Analysis of orphan drug procedures in early benefit assessment: RCTs versus best available comparative evidence

To assess the effectiveness and safety of a new active ingredient, the randomized controlled trial (RCT) is the gold standard for the European Medicines Agency (EMA). With the introduction of the AMNOG (Medicine Market Reorganization Act) on January 1, 2011, methodological aspects for appropriately answering the relevant question, namely the benefit assessment of innovative pharmacotherapies in accordance with Section 35a SGB V, came to the fore. In principle, clinical evidence from RCTs is used as the basis for evaluation when evaluating the relative effectiveness of a new drug compared to the appropriate comparative therapy (zVT). For drugs used to treat rare diseases (orphan drugs, OD), generating evidence using RCTs poses particular challenges. As a result, the special EU legal provisions for orphan drugs came into force in 2000 with the aim of promoting the research, development and marketing of suitable drugs by the pharmaceutical industry (Regulation (EC) No. 141/2000). Against the same background, special regulations also apply to the AMNOG procedure for orphan drugs, which implement the objectives of the European regulation at the national level (Section 35a Paragraph 1 Sentence 11 f. SGB V.). The additional benefit of orphan drugs is considered proven upon approval. On December 23, 2021, the Institute for Quality and Efficiency in Health Care (IQWiG) published working paper GA21-01, in which it critically examines the body of evidence and the benefit assessment of orphan drugs (Institute for Quality and Efficiency in Health Care (IQWiG) 2021 ). In the working paper, IQWiG makes a comparison between a benefit assessment of orphan drugs, which takes into account the special features upon market entry (hereinafter OD benefit assessment), and a regular benefit assessment. From its analysis in working paper GA21-01, IQWiG concludes that, even for orphan drugs, only randomized controlled studies with comparisons to the appropriate comparator therapy should be taken into account as the evidence basis for assessing the additional benefit.

methodology

>> The authors carried out their own analysis of the database extracted by IQWiG. For this purpose, the number of patients or patient groups eligible for treatment at the time of the initial OD benefit assessment, the number of patients in the respective studies relevant to the assessment, and the type of study (RCT , yes or no), the additional benefit (additional benefit, yes or no) in the respective procedure and the soloist status (soloist, yes or no) at the time of the respective procedure. The period between procedures was calculated in days from the G-BA decision data. For the regular benefit assessments, the question was also asked as to whether the appropriate comparator therapy was used within the procedure (zVT, yes or no). The statements postulated by IQWiG were subsequently checked based on the extracted data (Appendix). The results of this analysis are presented below.
Result
1) Orphan drug benefit assessment versus regular benefit assessment – different initial situations of the assessment procedures
If AMNOG-relevant orphan drugs exceed a sales limit of 50 million euros per year or the orphan drug status is returned, they go through another regular benefit assessment process. In contrast to the initial OD assessment process, which takes into account the regulatory peculiarities of the orphan drug approval (an additional benefit is considered proven upon approval), the additional benefit must be assessed as part of the regular assessment process compared to the appropriate comparator therapy in accordance with the methodological requirements of the G- BA can be demonstrated. In the IQWiG report GA21-01, the authors complain that the additional benefit granted at the time of market access of orphan drugs in the context of the OD assessment is often misleading, since later regular re-evaluations do not reveal any additional benefit compared to the appropriate comparator therapy would. An adequate representation of the therapy standard according to IQWiG depends centrally on the therapy situation at the time the regular second assessment is carried out. The treatment standard often changes in the period between the first and second assessment, so that the evidence from the initial assessment justifying the approval is no longer taken into account by IQWiG simply because it does not reflect the required comparator therapy that is considered appropriate. The median duration between the initial OD and regular benefit assessment procedure is 994 days (P25 581, P75 1477) and an average of 1146 days (SD 706.6) (Table 1, own calculation; IQWiG working paper GA21-01). This means that on average there is an average gap of almost 3 years between the two benefit assessments.
Furthermore, the IQWiG postulates in its working paper,
that a misleading fictitious additional benefit of orphan drugs disadvantages existing treatment options. Orphan drugs are characterized by the fact that at the time of approval they either have solo status or show a “significant benefit” compared to the current supply situation (Regulation (EC) No. 141/2000). This means that no other approved therapy is available in the relevant indication or that the evidence presented for the relevant orphan drug has been assessed by the European regulatory authority in terms of the benefit-risk profile in such a way that there is a significant benefit compared to previously available therapies for the patients.
In the underlying data set from IQWiG with 41 orphan drug procedures, 27 (65.9%) have sololist status at the time of the initial OD benefit assessment. This means that at this point in time there was no approved therapy for the respective rare disease (own calculation; IQWiG working paper GA21-01). In 8 of these 27 (29.6%) procedures with soloist status, the orphan drugs lost their soloist status at the time of re-evaluation (Table 2, own calculation; IQWiG working paper GA21-01). For regular orphan drug procedures whose soloist status was not lost, there is a 2.5 times higher probability that an additional benefit will be declared compared to procedures in which an additional therapy option is approved in the period between the first and second assessment became. In 6 of the 8 (75.0%) regular benefit assessments of orphan drugs in which an additional treatment option was approved between the first and second assessment, the G-BA did not grant any additional benefit.
It should be questioned that at the time of the subsequent full assessment (median after 994 days, average after 1,146 days) evidence must be available that is not only based on a direct comparative RCT, as required by IQWiG, but also at the time of the second procedure The current therapeutic standard is then taken into account in the sense of the appropriate comparative therapy. A direct comparative study against a treatment standard that has since been newly named (appropriate comparator therapy) is in fact not feasible at this time. It is precisely this fact that in 8 benefit assessments of the underlying 41 orphan drug procedures, no additional benefit could be proven for 6 procedures, since new therapies in the respective indication were brought onto the market between the two evaluation processes. This fact is not taken into account by IQWiG in its assessment. Assuming a fictitious additional benefit from market access seems inappropriate. The general question also arises as to how the 27 orphan drugs with soloist status (65.9%) can be described as having a fictitious additional benefit at the time of approval, since there is simply no approved treatment alternative available. The IQWiG's representation that there was a fictitious additional benefit at the time of market launch requires a differentiated, serious look.

2) Prevalence as a measure of the evidence base – what is the difference in the feasibility of an RCT?
The epidemiology of the disease and the size of the population of sufferers for whom therapy is an option play a critical role in the successful implementation of an RCT. The IQWiG generally concludes that RCTs are also possible for orphan drugs. A differentiated analysis of the database of the IQWiG working paper GA21-01 of the total of 41 orphan drug procedures shows that the number of patients eligible for the therapy according to the information from the respective G-BA resolution (target population) also corresponds to the Size of the study population is associated. So the larger the target population, the larger the study population (Spearman's correlation coefficient 0.7471, p-value < 0.001) (Table 3, own calculation; IQWiG working paper GA21-01).
The feasibility of an RCT is also associated with the size of the population. The larger the population that is eligible for a study, the easier it is to implement an RCT. It is therefore also logical that the size of the populations differs between benefit assessment procedures with and without RCTs. For this purpose, the 41 orphan drug procedures from the IQWiG paper were examined based on the number of patients eligible for the therapy in accordance with the G-BA resolution. In 30 (73.2%) evaluation procedures conducted based on one RCT, the mean population size was 2,445 patients (SD 467.5) with a median of 1,725 patients (P25 250 patients, P75 3550 patients), versus 471 patients (SD 193.1) and a median of 440 patients (P25 17 patients, P75 455 patients) in 11 (26.8%) procedures without RCT as a basis for evaluation. Since the data do not have a normal distribution (Shapiro-Wilk test, p-value < 0.001), the difference was assessed using the rank sum test. This showed statistical significance (p-value 0.0123) (Table 4, own calculation; IQWiG working paper GA21-01). This leads to the conclusion that the actual conduct of an RCT is statistically significantly associated with the size of the target population.
RCTs are carried out for rare diseases with a relatively high prevalence. Pharmaceutical companies do not fundamentally rule out conducting RCTs for rare diseases, but operational feasibility with small patient numbers in multicenter, international studies is still a major challenge. This once again confirms that a more differentiated look at the respective epidemiology of the rare disease and the implementation of an RCT is warranted. In the case of very rare diseases in which the number of potentially available and recruitable patients for clinical studies is obviously limited, alternative evidence bases with higher uncertainty in the results should be able to be considered for the benefit assessment process.
The same applies to orphan drug soloists. Of 27 procedures with soloist status at the time of the initial orphan drug assessment (Table 5), 17 (63.0%) had an RCT as the basis for evaluation, while 10 (37.0%) benefit assessment procedures were not based on RCT evidence. The number of patients eligible for therapy according to the information from the respective G-BA resolutions is on average 288 (SD 216.3) for the orphan drug soloists without RCT evidence and for the orphan drug soloists with benefit assessment relevant RCT 1,669 (SD 2,793.6) (Table 5, own calculation; IQWiG working paper GA21-01).
A population-focused analysis of the IQWiG data set shows that RCTs are also carried out on soloists. For the orphan drug soloists without an RCT, the mean target population is almost 6 times lower compared to the soloists with an RCT as an assessment basis, so that it must first be questioned whether an RCT should be considered due to this small number of patients with regard to study results. and recruitment duration would have been adequately implemented. If such studies appear to be generally feasible, it should also be taken into account that for a small study group with the same effect size, lower statistical power makes it considerably more difficult to prove an additional benefit. Furthermore, in such a small study population, randomization per se does not ensure an even distribution of possible confounding variables (so-called confounders) across the intervention arms.
3) Evidence base RCT – How can the appropriate comparative therapy be adequately implemented?
In the subgroup of 30 orphan drug procedures (Table 6), which were based on an RCT, no additional benefit was recognized in any procedure in which the appropriate comparator therapy required by the G-BA was not included in the RCT (10 of 30 procedures ). However, if the appropriate comparator therapy was used as part of an RCT (20 of 30 procedures, 66.7%), an additional benefit was awarded in 17 of the 20 (85.0%) procedures. In 13 of 30 (43.3%) orphan drug procedures, no additional benefit was declared despite an RCT-generated evidence base. Consequently, it can be stated that, in addition to the RCT evidence base, the decisive methodological criterion for deriving an additional benefit in regular benefit assessments is a direct clinical comparison with an active comparator arm, which is accepted by the G-BA as an adequate implementation of the ACT. In over half of the orphan drug procedures with RCT as the basis for evaluation (20 of 30 procedures, 66.7%), these requirements were met at the time of re-evaluation.
A closer look at the 20 orphan drug trials (Table 7) in which there was an RCT that also adequately reflected the zVT defined by the G-BA shows that these were mostly trials for which a placebo was used. The comparison arm represented an acceptable implementation of the ACT specified by the G-BA. This was operationalized via “best supportive care” or “watchful waiting” (13/20 procedures, 65%) or when a placebo as a comparator intervention represented an equally useful therapeutic option (“idelalisib or BSC” and “pirfenidone or BSC”; 2/20 Procedure, 10%). These were therefore procedures in which the pivotal placebo-controlled study represented the appropriate evidence basis in the context of the benefit assessment for the G-BA due to a lack of treatment alternatives. However, if the G-BA had specified a specific active ingredient as an appropriate comparator therapy in these 15 RCT-based procedures, in principle no additional benefit would have been detectable.
The 17 orphan drug procedures with attested additional benefits based on an RCT were further examined in more detail to determine the extent to which the number of patients in the target population and the size of the study populations differ between placebo-controlled and pharmacologically active-controlled studies. In 10 of the 17 (58.8%) assessment procedures, the mean size of the placebo comparison arm population was 1,650 patients (SD 3,163.97) and a median of 271.5 patients (P25 44 patients, P75 1,680 patients), while the mean Size of the population with active comparator in the RCT was 3,439 patients (SD 2,677) with a median of 2,300 patients (P25 1,200 patients, P75 5,850 patients) in 7 (41.2%) procedures (Table 8).
A comparable trend can also be seen when examining the study population. With an identical procedure, the mean size of the study population with a placebo control arm corresponds to 197 patients (SD 173.33) and a median of 140.5 patients (P25 52 patients, P75 265 patients) compared to a mean study population of 472 patients (SD 336 ,33) and a median of 455 patients (P25 122 patients, P75 792 patients) in the orphan drug procedures with active comparator intervention (Table 9). The difference in population sizes was statistically evaluated using the rank sum test (size of the patient population
p-value 0.0508; Size of the study population p-value 0.0708) (Tables 8 and 9, own calculation; IQWiG working paper GA21-01). Based on this evaluation, it can be emphasized again that, in addition to the actual implementation of an RCT, the incorporation of an active comparative intervention into the RCT design is associated with the size of the target population and thus the frequency of the disease.
Depending on the prevalence, the methodological questions arise according to the study type (single-arm or two-arm study) and the study design with regard to the comparison intervention (placebo or active comparator, if another treatment option is available). It becomes clear once again that the prevalence must also be viewed in a differentiated manner within orphan drugs and assessed according to its potential for actual evidence generation.
In working paper GA21-01, IQWiG fundamentally demands that the postulated methodological requirements also be applied to the initial assessment of all orphan drug procedures. Rare diseases are extremely heterogeneous in their prevalence and disease biology. A general requirement for RCTs and mandatory incorporation of a pharmacotherapeutic comparison substance for orphan drugs appears to be methodologically debatable following the revision of the IQWiG data set. In addition, it should be remembered at this point that the granting of an orphan drug status by the EMA requires that there is no alternative therapy option for the relevant orphan drug in the indication or that the relevant active ingredient represents a “significant benefit” compared to the standard of care . Regardless of this, IQWiG considers it fundamentally necessary to implement a placebo control arm or, depending on the therapeutic situation, the therapy that the EMA assessed as inferior into the study design in order to reflect the defined therapy standard for the health-relevant question.
Overall, when considering the IQWiG, the fact that the number of cases in the study group required to carry out an RCT and the duration of the study increases significantly is not taken into account. As a result, orphan drugs are available to patients with rare diseases much later in routine care. As an example, we refer to the case number estimate and study duration of the data collection accompanying the use of onasemnogene abeparvovec (Zolgensma®), indicated for the treatment of spinal muscular atrophy (SMA), a hereditary disease with a prevalence of 4 in 100,000 (AveXis EU Ltd. 2020). A number of 500 patients was calculated here in order to be able to quantify any additional benefit. This results in a study period of up to 10 years (Resolution of the Joint Federal Committee on the amendment of the Medicines Directive (AM-RL): Onasemnogene-Abeparvovec (spinal muscular atrophy; requirement for data collection and evaluations during use) (Joint Federal Committee (G- BA) 2021). The extent to which this instrument can be used adequately to close evidence gaps relevant to care remains questionable, especially since non-randomized evidence is also collected here. The IQWiG also recognizes this problem of the limited evidence base for data collection during use and encourages the implementation of pragmatic RCTs. Nevertheless, such concepts are only suitable if available therapy options already exist, which was not the case in the underlying IQWiG data set in 2/3 (27 of 41, 65.9%) of the procedures at the time of market access for those orphan drugs.
discussion
The rare diseases are extremely heterogeneous in their frequency, despite their defined characteristic of a prevalence of fewer than 5 affected per 10,000 people in the EU. Many questions in the methodological study design are associated with the frequency of the rare disease. The present analysis shows that conducting a study with an RCT or non-RCT design is associated with disease prevalence. Even within the RCT-generated evidence of orphan drug procedures regarding implementation as a placebo-controlled or active-controlled study, the analysis reveals the dependence of disease frequency. The IQWiG references cited, which are intended to demonstrate the fundamental feasibility of RCTs for rare and very rare diseases, do not specifically address the question of the comparator in the control arm (Naumann-Winter, Kaiser et al. 2022). Rather, it is also stated here that the mean study population in RCTs with a comparison against standard therapy was twice as large as the mean study population in RCTs with a placebo comparison (Hee, Willis et al. 2017). Given the fact that around 85% rare diseases have a prevalence of less than 1 in 1,000,000, which in Germany means a maximum of 80 patients in the statutory health insurance for whom a respective therapy would be an option, is the IQWiG to question the blanket demand for RCT-generated evidence (Nguengang Wakap, Lambert et al. 2020). For this small number of patients, the best available evidence should be accepted with recognition of the higher uncertainty in the results. For example, the evidence requirements could be differentiated according to prevalence categories to enable an assessment taking epidemiology into account. Different prevalence gradings could be defined that allow the acceptance of non-randomized studies, RCTs with a placebo control arm and RCTs with comparison to the available standard of care, if this is available at the time the study is carried out. The question of soloist status when entering the market should also be given sufficient consideration. In two thirds of the orphan drug cases, no other approved therapy was available in the relevant indication at the time of market launch.
IQWiG's finding that in more than half of the cases it is unclear at the time of market launch how a new orphan drug compares to the available therapy options cannot therefore be derived from the underlying data set. The discussion for new assessment approaches should take place not only with methodologists but also with clinicians and other specialists and should not be ideologically driven. Medical societies are already discussing focused funding for the development of drugs for rare diseases in indications with low incidence and prevalence (Association of Scientific Medical Societies e. V. (AWMF) 2022). The authors therefore advocate a dialogue with all relevant actors that allows a differentiated approach to orphan drug procedures and promotes compromise-oriented solutions that can also be taken into account in practical implementation for early benefit assessment. <<