Principles of benefit-risk assessment: A focus on some practical applications

Assessing benefit versus risk is integral to decision making throughout the life cycle of any drug/product. Over recent decades the method of conducting benefit-risk (BR) assessments has evolved from a non-structured and often mainly qualitative  process to one that has become increasingly systematic and more transparent [1]. In 1998, the seminal report by the Council for International Organizations of Medical Sciences (CIOMS) was produced, entitled “Benefit–risk balance for marketed drugs: evaluating safety signals” [2]. The authors acknowledged that there was no structured and harmonized approach to BR assessment at that time. The report also included reference to how benefits and risks could be scored (or ranked) based on the properties of each outcome.

In the early 2000s two papers generated additional interest in quantitative BR methods using weighted outcomes [3, 4]; these were followed by numerous initiatives to explore BR methodologies further. In 2006 the Pharmaceutical Research and Manufacturers of America (PhRMA) developed a structured framework: the Benefit Risk Action Team (BRAT) Framework [5]. A few years later, the European Medicines Agency (EMA) began a three-year “Benefit-risk methodology project” with the aim of identifying decision-making models to improve consistency and transparency. In the 2012 report from this project, “Work package 4 report: Benefit-risk tools and processes,” it was suggested that application of the PrOACT-URL model (a descriptive framework) may be sufficient. PrOACT-URL is a generic decision-making guide with eight steps: Problems, Objectives, Alternatives, Consequences, Trade-offs, Uncertainty, Risk attitudes, and Linked decisions. The framework itself is generic and can be applied to any decision-making problem, but the EMA’s Benefit-Risk Project adapted PrOACT-URL to potential use for decision-making in medicines [6] . Alternatively the report suggested that development of a multi criteria decision analysis (MCDA) model may be useful for difficult or contentious cases, i.e. when the BR balance is marginal or in the case of many conflicting attributes [7, 8].

In 2009 the FDA began an initiative to develop a structured method for BR assessments, and concluded that a qualitative descriptive approach could be sufficient while acknowledging that quantification of certain components of the assessment was important to support decision-making [9]. The PROTECT project (2009-2015) funded by the Innovative Medicines Initiative (IMI), was a public-private partnership between the European Union and the European Federation of Pharmaceutical Industries and Associations (EFPIA) and aimed to address limitations of current methods used in pharmacovigilance and pharmacoepidemiology and to significantly strengthen the monitoring of benefit-risk of medicines marketed in Europe [10]. One of the BR methods used in the PROTECT project and applied to various case studies was the BRAT framework [11].  This is a descriptive framework, based on six main steps which are shown below [5]. In this framework the individual benefits and risks are not integrated per se but are assessed separately and clearly, so that a comparison can be more readily achieved. This was a conscious proposal to avoid synthesising data into complex statistical models [10].

In response to the ongoing pandemic and the urgent need to identify effective and safe treatments for COVID-19 disease, the Drug Safety Research Unit (DSRU, UK) conducted four BR assessments on potential COVID-19 therapies (remdesivir, lopinavir/ritonavir, hydroxychloroquine, convalescent plasma) [12-15]. All of these BR assessments were conducted in the context of treatment of patients with severe COVID-19 disease (i.e. amongst those patients who required hospitalisation). The underlying approach was based on the BRAT framework, however where sufficient data was available further quantitative analysis was conducted.

As with similar descriptive frameworks, the key anticipated benefits and risks associated with each drug were identified by clinicians with pharmacovigilance expertise, based on literature search findings (including clinical trial protocols for relevant study endpoints, especially relevant for identifying key anticipated benefits of treatment). Key outcomes refer to those outcomes likely to impact the BR assessment, which are then visually presented in the form of a value tree, in which both the benefits and risks are ranked (separately) based on levels of seriousness.

Where data was available for each risk and benefit from controlled studies, this was summarised and differences in risk calculated between the drug of study and the comparator group (placebo, standard of care, other treatment). As mentioned above, this framework also lends itself (where enough data is available) to perform some additional “add-on” quantitative analysis which may be helpful where there are multiple outcomes of interest involved or the difference is considered marginal. To facilitate this additional quantitative analysis, both benefits and risks are ranked on a single continuum and preferential weights are applied for each outcome. This forms the basis of many quantitative analyses, i.e. the combination of weighted benefits and risks to produce an overall metric. As time goes on and more study results are made available for the ongoing COVID-19 potential treatment options, it is possible to calculate the weighted net clinical benefit (wNCB). In this approach,  benefits have a positive contribution to the wNCB and risks have a negative contribution; the overall wNCB is positive (benefit outweighs the risk) where wNCB is >0 [16].

Finally, how frequently should BR assessments be conducted? For approved medicines there are well defined regulatory milestones during which BR is formally assessed, with the assumption that overall BR is being continually monitored. At other times however, such as during the COVID-19 pandemic, treatments are given to patients based on very little or no evidence either in the absence of findings from more robust randomised studies or because these results take time to accrue. In a fast-changing environment such as this, how often should systematic BR assessments be reassessed to incorporate new or changing evidence as this becomes available? The acknowledgment that BR is based on a living body of evidence and as such is a dynamic evaluation that will change over time is a fundamental principle of BR assessment. Essentially, updated BR assessments should be produced for those drugs under ongoing study as more data becomes available, facilitating the addition of supportive quantitative analyses. The increasing transparency and structure seen in assessing BR allows for a clearer understanding of this process by all parties and likely facilitates more meaningful comparison between various treatment options.