Perception of the need for an appropriate benefit-risk framework
The results were collated from the responses to 13 statements in the study regarding the perception of the need for an appropriate framework. Eleven agencies and 20 companies responded and these responses were reviewed and presented as three categories namely utility and scope, purpose and direction for developing a benefit-risk framework.
Utility and Scope of a benefit-risk framework
Most agencies felt that a benefit-risk framework should be used by both agencies and companies, across divisions of a regulatory agency, and be applied from drug development to post-approval changes (Figure 3.6). Responses from the companies had a similar trend.
Fewer agencies believed that the framework, if developed for registration of medicines, should be utilised across agencies worldwide. However, the majority of companies would prefer this to be so. It was also observed that more companies than agencies wanted the framework to be applicable to heath technologies agencies (HTA).
Figure 3.6 Responses to perceived utility and scope of a benefit-risk framework
The general consensus was for a benefit-risk framework to be utilised by both agencies and companies and for the entire life cycle of a medicine.
Purpose of a benefit-risk framework
There was a good level of agreement between the agencies and companies for the purposes of a framework. Both groups felt that a benefit-risk framework would enhance the quality of communication and enable the assessment of benefit-risk management plans (Figure 3.7).
65
Important that any benefit-risk framework if developed for registration purpose is utlised
across agencies worldwide
Benefit-risk framework should also be applicable to Health Technology
Assessment groups Appropriate benefit-risk framework for registration should also apply to all stages
of drug development from drug development to post-approval changes Important to any benefit-risk framework, if
developed for registration purposes, is utilised across regulatory divisions within
agency
There is a need for benefit-risk framework to be developed that can be used by both
agencies and companies
Percentage of responders
Perceived utility and scope
Agencies Companies
63
Similarly, both agencies and companies did not feel the need to have a framework that translates benefit-risk balance into absolute numeric terms and measures sensitivity to various other parameters. This closely mirrored the observations that no responders currently utilise a fully quantitative system and the inconsistent use of values and weights for benefit-risk parameters.
Figure 3.7 Responses to perceived purposes of a benefit-risk framework
Direction for developing a benefit-risk framework
A high proportion of the agencies would prefer a quantitative approach in assessing benefits and risks and have an overarching framework (Figure 3.8). Majority of the companies would prefer to have a coordinating group (consisting of representatives
25
90 95 95 95
36
82 73
91 82
0 50 100
Purpose of appropriate benefit-risk framework is to define number that translate benefit-risk balance in absolute
terms and can be used to measure its sensitivity to various parameters Purpose - Communication of decision Purpose - Consistency of decision making Purpose - Transparency of decision making
Appropriate benefit-risk framework for registration should also enable assessment
of benefit-risk management plans
Percentage of responders
Perceived purposes of a framework
Agencies Companies
64
from agencies, companies, academia and other stakeholders) to guide the direction and application of the framework and to involve these relevant stakeholders in developing and validating the framework. These outcomes were agreed by both agencies and companies. Differences in opinions could be observed in the preference for a quantitative approach, and the need to develop specific frameworks for different therapeutic areas.
Perceived advantages of benefit-risk framework
This study evaluated the perceived advantages of a framework through nine statements. All responders, 11 agencies and 20 companies, provided responses to this section. The main advantages of a benefit-risk framework, as perceived by agencies, were in providing documentation for a structured discussion, acting as a tool for communication among peers within the organization and communicating between the organization and stakeholders (Figure 3.9). The main advantages, indicated by companies, were to enhance transparency and accountability and communicate between the organization and stakeholders.
A major discrepancy between the agencies’ and companies’ responses was in having the framework as a training tool with more than half of the agencies believing this advantage was significant, but not with the companies. Among the responders, all the listed advantages were considered significant. Between agencies and companies, there was a general agreement that the advantages of a framework included proper documentation and enhancement of communications (including transparency and accountabililty of decisions). The advantage of streamlining of current work did not appear to be a high priority. Additional comments received from these responders included the advantages of focusing on both benefits and risks of a medicine as well as providing a tool for decision-making in urgent situations.
65
Figure 3.8 Responses to the perceived directions in developing a benefit-risk framework
41
60 65
75 85
89 36
82 64
73
0 20 40 60 80 100
Preference would be a quantitative approach to benefit-risk assessment rather than a purely qualitative approach For registration of new medicinal product it will be necessary to
develop therapeutic area specific benefit-risk frameworks For registration of new medicinal product it will be possible to
develop an overarching benefit-risk framework
Important that all stakeholders are part of the development and validation of an appropriate benefit-risk framework There is a need for a coordinating group including representatives
from agencies, academia, pharmaceutical companies, and other relevant stakeholders to ensure appropriate direction and application of benefit-risk systematic standardised framework
Percentage of responders
Perceived directions in developing a framework
Agencies Companies
66
Figure 3.9 Responses indicating the perceived advantages of a benefit-risk framework
Perceived advantages of a benefit-risk framework
Agencies Companies
67
Factors for reviewing benefit-risk frameworks
The major factors for reviewing a benefit-risk framework were logical soundness, acceptability of results and practicality. These results were similar for both agencies and companies (Figure 3.10).
Figure 3.10 Responses indicating the relevance of factors for reviewing a benefit-risk framework
In general, all the listed factors could be considered relevant in reviewing a benefit-risk framework for appropriateness. Additional comments provided by responders were to include factors like transparency of the methodology and provision of an audit trail from evaluation to decision.
55
70 60
85 80
90 75
45 55
64 64
82 82
100
0 20 40 60 80 100
Specificity and sensitivity Comprehensiveness Scope Presentation/Visualization Practicality Acceptability of results Logical soundness
Percentages of responders
Factors for reviewing a framework
Agencies Companies
68
Criteria in constructing benefit-risk balance
The criteria used for constructing a benefit-risk balance were similar between the agencies and companies. The more frequently used criteria were the description of alternative therapies or interventions, the identification of outstanding issues and potential post-market commitments (Figure 3.11). In addition, other criteria included the direct comparisons of the absolute gains or harms in terms of lives saved, lost, or specific clinical events. Five out of 11 agencies (45%) and three out of 20 companies (15%) calculated the benefit-risk balance for each major subpopulation. Similarly there was a difference with respect to the acceptable level of risk with regards to clinical benefit (36% of agencies compared with 16% of companies) and the evolution of benefit-risk balance over time (36% of agencies compared with versus 20% of companies). The remaining criteria, namely consideration for different regulatory options for approval and calculation of the uncertainties for benefit and risk were used in similar frequencies by agencies and companies.
In considering criteria important to construct a benefit-risk balance, there was agreement between the agencies and companies to include the calculation of uncertainties on benefits and risks, direct comparison of absolute gains or harms, calculation of acceptable risk with regards to clinical benefits, the description of alternative therapies or interventions and the identification of outstanding issues and potential post-market commitments (Figure 3.12). With the exception of the calculation of acceptable risk with regards to clinical benefits, the rest were currently used in similar frequencies by agencies and companies. In general, these five criteria should be considered in the development of a benefit-risk framework.
69
Figure 3.11 Comparison between agencies and companies for criteria currently used in constructing benefit-risk balance
16
Calculation of level of risk would be acceptable with regards to level of clinical benefit in the specific context Evolution of benefit-risk balance over
time and its sensitivity to various assumptions
Calculation of the uncertainties on benefit and risk
Consideration of different regulatory options for approval (standard,
conditional, priority)
Calculation of benefit-risk balance for each major patient subpopulation Direct comparison of the absolute gains or harms in terms of lives saved
or lost, or in terms of specific clinical events
Identification of any outstanding issues and potential post-market
commitments in this regard Description of alternative therapies or
interventions ie clear description of medical need
Percentage of responders
Criteria currently used in constructing benefit-risk balance
Agencies Companies
70
Figure 3.12 Comparison between agencies and companies for criteria considered important to be included in constructing benefit-risk balance
There was a difference between agencies and companies with respect to the criteria as to whether the benefit-risk framework could be of value for regulatory approval options. Half the companies reported that this criterion was important, whereas in contrast the agencies considered it to be of no value. Other differences were also
47 Calculation of benefit-risk balance for
each major patient subpopulation Description of alternative therapies or
interventions ie clear description of medical need
Evolution of benefit-risk balance over time and its sensitivity to various
assumptions
Calculation of level of risk would be acceptable with regards to level of clinical benefit in the specific context
Direct comparison of the absolute gains or harms in terms of lives saved or lost, or in terms of specific
clinical events
Calculation of the uncertainties on benefit and risk
Percentage of responders
Criteria considered important to be included in constructing benefit-risk balance
Agencies) Companies
71
observed for two other criteria namely evolution and sensitivity of benefit-risk balance over time and calculation of benefit-risk balance for each major patient subpopulation, with more companies considering them important to be included.
Part III – Barriers and Solutions to Implementing Benefit-risk Assessment