Zum Inhalt springen
Sponsorship
Die MVF-Knowledge-HubsFrisch aus der Forschung. Täglich aktualisiert!

Suchen Sie in Hunderten nationaler und internationaler Datenbanken nach dem neuesten Studienwissen der angegebenen Themengebiete und/oder lassen Sie sich werktäglich per Newsletter informieren. Die Newsfeeds werden auf deutsch übersetzt and nach Übertragbarkeit auf das deutsche Gesundheits-/Versorgungssystem ausgewählt:

Zu den Knowledge-Hubs

Application-accompanying data in the benefit assessment

The following concept is intended to contribute to the methodological discussion about the use of application-related data in benefit assessment. Based on an overview of the current handling of non-randomized data by relevant health technology assessment (HTA) organizations as well as an overview of possible methodological evaluation procedures, recommendations are formulated on evidence generation, evidence assessment and necessary G-BA specifications that must be taken into account of the data for the quantifiability of an additional benefit.

Read more
First publication date: 04.04.2019

Abstract: Anwendungsbegleitende Daten in der Nutzenbewertung

Background: With the GSAV, the G-BA is authorized to demand application-accompanying data collection and analyzes for the purpose of benefit assessment for orphan drugs and medicinal products with conditional approval or approval granted under special conditions. According to the intention of the legislature, non-randomized studies and thus lower certainty of results are also to be accepted.Methodology: Based on an overview of the current handling of non-randomized studies by relevant organizations and an overview of methodological evaluation procedures, recommendations regarding the generation and evaluation of evidence are formulated , which serve the objective of being able to generate usable data for a quantifiable additional benefit and thus to improve the evidence situation.Results: In a step-by-step model for selecting the appropriate type of evidence generation, prospective comparative cohort studies, historical comparisons or pre-post comparisons are used under certain conditions considered feasible and appropriate, with the studies commissioned by the EMA being given priority in this selection. Internationally recognized evaluation methods for confounder adjustment are considered suitable for being able to provide sufficient certainty of results for statements on the additional benefit, even without further requirements for the effect size (in particular “dramatic effect”). In the case of statistical tests, an increased probability of error should also be accepted if possible limitations in the number of cases to be expected are foreseeable within a reasonable study period. Internationally accepted methods of evidence assessment from leading EbM actors such as GRADE and Cochrane can provide a suitable framework for the acceptance of the data. Conclusion: The intention of the legislator to also use non-randomized studies to quantify an additional benefit as part of the application-accompanying data collection can be implemented with the proposed concept. It sets a possible framework in which data and evaluation can be used for a quantifiable additional benefit in order to improve the evidence base. However, binding advice on all essential aspects of data generation and evaluation is required to ensure that complex data collection is avoided, which is not expected to contribute to a (quantifiable) additional benefit.

Abstract: Using application-accompanying data for benefit assessments – Recommendations for evidence generation and evaluation

Background: The law for higher safety in the supply of medicines (GSAV) authorizes the G-BA to request application-accompanying data collection and evaluations for the purpose of benefit assessments for orphan drugs, products with conditional authorization and drugs authorized under exceptional circumstances. In accordance with the intention of the legislator, non-randomized studies and thus lower certainty of results has to be accepted.Methodology: Based on an overview of the current handling of non-randomized studies by relevant organizations and an overview of methodological evaluation methods, Recommendations are formulated with regard to the generation and evaluation of evidence, which serve the purpose of being able to generate usable data for a quantifiable additional benefit and thus the evidence base improve.Results: In a step-by-step model for selecting the appropriate type of evidence generation, prospective comparative cohort studies, historical comparisons or pre-post comparisons are considered feasible and suitable under certain conditions, whereby the studies commissioned by the EMA should be given priority in this selection. Internationally recognized evaluation methods of confounder adjustment are considered suitable to be able to provide a sufficient certainty of results for statements about the additional benefit, even without further requirements regarding the effect size (esp. “dramatic effect”). In the case of tests statistical, an increased probability of error should also be accepted if possible limitations in the expected number of cases can be foreseen within a reasonable period of study. Internationally accepted methods of evidence assessment by leading EbM players such as GRADE and Cochrane can provide a suitable framework for the recognition of the data.Conclusion: The intention of the legislator to use non-randomized studies to quantify an additional benefit in the context of the application-related data collection can be implemented with the proposed concept. It sets a possible framework in which data and evaluation can be used for a quantifiable additional benefit in order to improve the evidence base. Binding advice regarding all essential aspects of data generation and evaluation is required, however, so that it can be ensured that time-consuming data collection is avoided that cannot be expected to contribute to a (quantifiable) additional benefit.

Literature

[1] The Commission of the European Communities. (2003). Commission Directive 2003/63/EC of June 25, 2003 amending Directive 2001/83/EC of the European Parliament and of the Council on the Community code relating to medicinal products for human use. Brussels; 2003. [2] European Parliament and the Council of the European Union. (2000). Regulation (EC) No 141/2000 of the European Parliament and of the Council of 16 December 1999 on orphan medicinal products. Official J Eur Communities, 43, L18. [3] European Medicines Agency. (2010). Recommendation on elements required to support the medical plausibility and assumption of significant benefit for an orphan designation. (EMA/COMP/15893/2009). London; 2010. [4] German Bundestag (2010). 17th electoral term. Federal Government draft law - Draft of a law to reorganize the pharmaceutical market in statutory health insurance (Medicine Market Reorganization Act - AMNOG). Printed matter 17/3116. Berlin; 2010. [5] European Medicines Agency. (2006). Guideline on the Scientific Application and the Practical Arrangements Necessary to Implement Commission Regulation (EC) No 507/2006 on the Conditional Marketing Authorization for Medicinal Products for Human Use Falling Within the Scope of Regulation (EC) No 726/2004 (EMEA/509951/ 2006). London; 2006. [6] European Medicines Agency. (2005). Guideline on Procedures for The Granting of a Marketing Authorization under Exceptional Circumstances, Pursuant to Article 14 (8) of Regulation (Ec) No 726/2004 (EMEA/357981/2005) London; 2005. [7] European Medicines Agency. (2015). Scientific guidance on post-authorization efficacy studies. (EMA/PDCO/CAT/CMDh/PRAC/CHMP/261500/2015). London; 2015. [8] European Medicines Agency (2018) Discussion paper: Use of patient disease registries for regulatory purposes – methodological and operational considerations. London; 2018. [9] Behring, A. (2018). Healthcare assessment – blessing or curse. Drug evaluations from the perspective of the G-BA. Lecture at the Barmer Healthcare Research Congress 2018. Berlin; 2018. [10] Federal Joint Committee. (2018). Statement by the impartial members of the Federal Joint Committee (G-BA) of December 14, 2018 on the hearing of the Federal Ministry of Health on the draft bill for a law for greater safety in the supply of pharmaceuticals (GSAV). Berlin; 2018. [11] Institute for Quality and Efficiency in Healthcare. (2015). Statement by IQWiG on the draft bill of the Federal Ministry of Health. Draft of a law for greater safety in the supply of pharmaceuticals (GSAV) from November 14, 2018. Cologne; 2018. [12] Institute for Quality and Efficiency in Healthcare. General methods. Version 5.0. from July 10, 2017. Cologne; 2017. [13] Balshem, H., Helfand, M., Schünemann, HJ, Oxman, AD, Kunz, R., Brozek, J., ... & Guyatt, GH (2011). GRADE guidelines: 3. Rating the quality of evidence. Journal of clinical epidemiology, 64(4), 401-406. [14] Jüni, P., Loke, YK, Pigott, TD, Ramsay, CR, Regidor, D., Rothstein, HR, ... & Shrier, I. (2016). Risk of bias in non-randomized studies of interventions (ROBINS-I): detailed guidance. [15] European Network for Health Technology Assessment (2019). Vision paper on the sustainable availability of the proposed Registry Evaluation and Quality Standards Tool (REQueST). EUnetHTA JA3 WP5B2: Vision paper. Copenhagen; 2019. [16] Federal Joint Committee. (2019). Main reasons for the decision of the Federal Joint Committee to amend the Medicines Directive (AM-RL): Appendix XII - Decisions on the benefit assessment of medicines with new active ingredients in accordance with Section 35a SGB V Vestronidase alfa. Berlin; 2019. [17] Institute for Quality and Efficiency in Healthcare. (2014). IQWiG reports – No. 241. Assessment and evaluation of studies in rare diseases. Rapid report. Cologne; 2014. [18] Federal Joint Committee. (2018). Main reasons for the decision of the Federal Joint Committee to amend the Medicines Directive (AM-RL): Appendix Berlin; 2018. [19] Federal Joint Committee. (2016). Main reasons for the decision of the Federal Joint Committee to amend the Medicines Directive (AM-RL): Appendix Berlin; 2016.

Additional Information

Citation: Bleß, H.: "Application-accompanying data in benefit assessment - recommendations for evidence generation and evaluation", in: "Monitor Health Services Research" (01/20), pp. 47-54, doi: 10.24945/MVF.01.20.1866-0533.2202

Plain Text

Application-related data in the
Benefit assessment - recommendations for generating and evaluating evidence

The following concept is intended to contribute to the methodological discussion about the use of application-related data in benefit assessment. Based on an overview of the current handling of non-randomized data by relevant health technology assessment (HTA) organizations as well as an overview of possible methodological evaluation procedures, recommendations are formulated on evidence generation, evidence assessment and necessary G-BA specifications that must be taken into account of the data for the quantifiability of an additional benefit.

>> The European Medicines Agency (EMA) generally (“as far as possible”) provides for the use of RCTs in the approval process. A non-randomized design when testing medicinal products is in principle possible, but is subject to a special obligation to provide justification [1]. For certain medicines in defined care situations, the EMA has established special approval pathways that (partially) enable access to care based on lower levels of evidence.
Regulatory framework
A medicine can be granted orphan drug status by the EMA if it is used in life-threatening, rare diseases for which there is no satisfactory treatment option. If treatment options are available, the new drug must demonstrate a significant benefit [2]. The term “significant benefit” is defined as a clinically relevant advantage or a significant contribution to the treatment of patients [3]. The explanatory memorandum to the Medicines Market Reorganization Act (AMNOG) on the exemption for orphan drugs therefore states: “Medicines that are […] designated as drugs for orphan diseases already provide evidence of their additional benefit through this designation.” [4].
Rapid access to new therapies should be made possible, particularly for patients with life-threatening illnesses. For example, a so-called “conditional approval” can be granted for the medicinal product in question, with which market access is possible before the full clinical trial has been completed. Conditional approvals are valid for one year and are subject to conditions and close monitoring until regular approval is granted. As part of these requirements, open questions about the quality, safety and effectiveness of the drug must be answered. The decisive factor for this approval route is that the public health benefit of the immediate availability of the drug outweighs the risks arising from the fact that additional data are still required [5].
Medicines for which comprehensive data cannot be provided for legal or ethical reasons are eligible for approval in exceptional circumstances. This type of approval is granted with conditions and the benefit-risk ratio is reassessed annually, but does not normally lead to the completion of a complete approval dossier [6].
Practical handling of non-randomized data
EMA accepts non-randomized data after approval under conditions
The EMA discusses in several places the use of non-randomized studies to generate further evidence after approval has been granted. The methodological considerations made there are the basis for any requirements that the EMA may impose in order to answer questions that remain unanswered about the effectiveness and safety of drugs after they have entered the market.
In the context of post-authorization efficacy studies (PAES), the EMA favors comparative study designs with patient groups that are similar in terms of the course of the disease. In addition, the EMA considers it desirable to identify a subpopulation in the observational study population that is broadly similar to that in existing RCTs. If similar results are found for these populations, confidence in the overall study results would be increased. Sensitivity analyzes to test the robustness of study results are therefore given great importance [7].
The comparison with historical datasets is an option for the EMA when it is not possible to obtain prospective data on concurrent controls. Possible differences between the populations to be compared, which affect the treatment decision and a clinical background that may have changed over time, are identified as a challenge. A good characterization of the historical data with regard to measured and documented selection criteria as well as known prognostic variables and their comparison with the observational study is important here.
Furthermore, pre-post comparisons are also possible, in which a patient is their own control for comparisons before and after treatment.
For application-accompanying registries, the EMA recommends the connection to existing disease registries in order to "enable a continuous evaluation of the disease results and a comparison of different treatment options according to a similar methodology" [7]. In its “Discussion paper registries”, the EMA makes clear the preference for disease-related over product-related registries [8]. The EMA sees the need to avoid and control selection bias as a key point. In support, the EMA offers to provide tools to facilitate the recognition of disease registries as a data source for conducting studies for regulatory purposes. At the same time, the EMA promotes early dialogue with the supervisory authorities and offers scientific advice on study protocols and the qualification procedures for registers [8].

Consideration of non-RCTs in special cases corresponds to G-BA judgment practice
The suitability of register data for the benefit assessment was already being discussed in the run-up to legislation. As a contribution to the discussion, the G-BA established the following criteria:
“Essential requirements for register data as a basis for discussion:
I. Generation of comparative evidence must be guaranteed
• Data from alternatively treated patients are also recorded using indication-specific registries
• Data collection begins before planned market access (control group)
II. Assessment of disease symptoms and quality of life is essential
III .Registers should examine at least a representative sample of the statutory health insurance population; in the case of orphan drugs, they should generally conduct a full survey
IV. Compatibility with existing registers (avoidance of double entries) [9]
In its statement on the GSAV, the G-BA points out that the use of non-randomized studies applies to drugs in indications “in which it is impossible or inappropriate to carry out or require studies of the highest level of evidence” [10]. At the same time, he demands that controlled studies should also be included in the data collection accompanying the application.
In its judgments in the past, the G-BA has also used non-randomized studies to recognize an additional benefit. In several proceedings, the time limit of the decision was also subject to the generation of register data, which is therefore generally considered usable for the benefit assessment.

IQWiG considers GSAV data collection to be unsuitable for assessing the additional benefit
In the IQWiG statement on the GSAV draft bill, the institute writes in the introduction that it considers application-accompanying data collection in the planned form to be unsuitable [11]. The IQWiG emphasizes that in order to eliminate knowledge deficits in the benefit assessment, direct comparative, randomized, controlled studies should primarily be used. IQWiG unequivocally rejects the study types provided for by the legislature:
• “Application observations are by definition unsuitable for the required comparative statements”
• “Case-control studies are […] “completely inappropriate for the questions relevant here”
• registers are time-consuming to set up; With non-comparative product registers, “comparative statements as a basis for assessing additional benefits will not be possible” [11].

In its current method paper 5.0, the institute states that “study types other than RCTs [..] are generally not suitable for proving causality” [12]. In principle, non-randomized comparative studies would not be a guarantee of structural equality between the groups and would therefore always provide a potentially distorted result. According to IQWiG, the use of non-randomized studies as proof of the causality of an intervention requires "special justification or special prerequisites and special quality requirements" [12]. The Institute recognizes exceptions in the case of
• Presence of “dramatic effects”, which means a reversal of a previously uninfluenceable deterministic course of the disease through an intervention.
• extremely rare diseases or extremely specific disease constellations for which the requirement for (parallel) comparative studies is inappropriate. Historical comparisons would be applicable here [12].
Achievability of high levels of evidence for observational studies using the GRADE approach is possible
The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system is an internationally widespread and accepted concept for evaluating evidence as well as for developing recommendations for action in the healthcare system. Although the GRADE approach serves to assess bodies of evidence as a whole, it also uses an assessment of the level of evidence at the study or outcome level.
A distinction is made between four levels of evidence (high, medium, low, very low). Randomized studies are initially accepted as evidence of high quality, while observational studies are initially classified as evidence of low quality. Both study types can be downgraded or upgraded in quality based on the initial classification, depending on their nature. As a result of this assessment, an observational study that has been upgraded twice can achieve the same quality level as an RCT. Conversely, bodies of evidence based on RCTs should not be rated higher in the presence of downgrading deficiencies than when they are based on high-quality observational studies [13].

Cochrane tool (ROBINS-I) is suitable for evaluating non-RCTs
With ROBINS-I (“Risk Of Bias In Non-randomized Studies - of Interventions”), the Cochrane Collaboration has developed a tool for the reliable assessment of bias in non-randomized studies [14]. Conceivable sources of bias that may be present before, during or after the intervention are assessed separately and lead to an overall assessment of the study.
Before the intervention, prognostic confounders and the risk of bias must be assessed. To identify potential confounding variables, collaboration with subject matter experts and the use of literature-based knowledge are recommended. As a result, the risk of bias of a study is evaluated in one of the categories “low risk”, “moderate risk”, “severe risk” and “critical risk of bias”. The “low risk” category corresponds to the risk of bias in a high-quality randomized study.

EUnetHTA defines uniform requirements for registers and enables them to be evaluated
As part of the European network for the assessment of health technologies (EUnetHTA), REQueST (Registry Evaluation and Quality Standards Tool) was developed, a tool that is intended to support the European Health Technology Assessment (HTA) agencies in setting up and evaluating registries [15 ]. The tool allows for a consistent assessment of the suitability of registries for HTA while addressing concerns about the reliability of registry data for these purposes. The tool is suitable for evaluating the suitability of registers and at the same time presents the criteria for a newly established, high-quality register in a comprehensible manner [15].
Possible types of evidence
So far, IQWiG and G-BA have argued against the consideration of non-randomized studies in the benefit assessment required by the GSAV that they are not suitable for the presentation of causal relationships due to the lack of randomization, since unknown confounding factors are not controlled.
Although this argument is not wrong in principle, it ignores the fact that any supposedly causal statement from studies contains both uncertainty in the results and the probability of error. Accordingly, the AM-NutzenV provides for different degrees of certainty of results, which must be determined at study level or at the level of the respective endpoint. In addition to the “evidence”, “indications” or “indications” are also recognized for the recognition of an additional benefit.
The use of non-randomized studies for the assessment of the additional benefit is therefore associated with a lower certainty of results at study level compared to RCTs. However, this fact does not fundamentally speak against the use of such studies. Rather, it involves the task of keeping the error as low as possible by using special methods and estimating as best as possible the extent to which remaining, distorting factors influence the study results.

Prospective comparative cohort studies are generally suitable for benefit assessment
A cohort is a group of people who are observed over a longer period of time. In a cohort study, groups are compared that are exposed to various influences, such as the use of a new therapy option. An inter-cohort comparison, in which the members of different cohorts are compared with one another, basically enables statements to be made about causal relationships and therefore represents an applicable study design for benefit comparisons. A prospective design is suitable for including new drugs on the market. The events that are to be measured, as well as possible influencing factors, must be described precisely in advance.
The completeness of product-related registers can be largely ensured through the statutory prescription restriction, even if the patient's consent to data use can be withdrawn. However, the disease-related registers also mentioned by law cannot guarantee completeness, as there is no legal basis for an obligation to collect data for therapy with therapy options not affected by the benefit assessment. In addition, it can be assumed that even in a complete, disease-related registry there is a selection bias that is difficult to control between patients who are offered a novel therapy option and patients who are treated according to the previous standard.

Inclusion of non-comparative studies is also necessary
The EMA also grants approvals based on single-arm studies in the special situations covered by the new legislation. The adjusted indirect comparisons using common comparators, which can be used in principle according to the IQWiG method paper and the G-BA’s rules of procedure, are not possible on the basis of this data. In addition, the special features in the approval covered by the proposed legislation relate to situations in which there is no adequate alternative therapy. As a result, situations will arise in which only a historical comparison with other study arms or a comparison with the natural course of the disease can be made.
Comparisons possible on the basis of single-arm studies
The challenge with such comparisons is to compensate for differences in the patient populations with regard to potentially confounding factors or even differences in time when the study was carried out. A historical comparison basically represents an unadjusted indirect comparison. In order to be usable - due to a lack of randomization - sufficient similarity of the studies must be guaranteed, particularly with regard to the population and the operationalization of the relevant endpoints.
In order to carry out historical comparisons, it is fundamentally necessary to use data from the time before the introduction of the drug to be evaluated. Whether and in what quality such data is available should be taken into account for the G-BA's decision on application-related data collection, but it can no longer be influenced at this point. If no suitable data is available for a historical comparison, pre-post comparisons can be used to quantify the additional benefit. The ruling practice of the G-BA shows the fundamental usability of such comparisons [16].
Carrying out pre-post comparisons should only be considered if, on the one hand, no historical comparison is possible and, on the other hand, based on knowledge of the course of the disease, it cannot be expected that a (fictitious) untreated control group will experience an improvement in health or quality of life would.
Possible evaluation methods
Statements about additional benefit are possible with suitable adjustment procedures
In order to guarantee the similarity of the characteristics of the comparison groups in a prospective cohort study or a historical comparison, procedures are required that allow adjustment for distorting factors. The basic prerequisite for such methods is the knowledge and consideration of essential distorting factors such as prognostic factors, disease characteristics or demographic factors. In contrast to randomization, however, unknown confounders cannot be recorded with this adjustment method.
There are several fundamentally suitable methods available with which sufficient feature similarity can be achieved by adjusting for different patient characteristics. Common methods are matching-adjusted indirect comparisons (MAIC) or propensity score matching (PSM), in which patient characteristics in the intervention arm are adjusted to those in the control arm by weighting. Methods such as Simulated Treatment Comparison (STC) or Bayes Benchmarking Analysis (BBA) are used less frequently. Despite the use of a suitable adjustment procedure, the certainty of results is objectively lower than that of a randomized study. However, their acceptance is intended by the legislator's stipulations for taking non-randomized data into account and is justified by the fact that the benefit assessments to be carried out are based on the EMA's approval decisions for this special group of medicines. In principle, an adequate adjustment in this situation enables valid statements on the added benefit.

Adjustment of the probability of error can be carried out at low
number of patients will be necessary
The level of the accepted probability of error should generally be taken into account when considering suitable evaluation methods. The usual use of a 95%igen confidence interval involves accepting an error level at 5 %
(Probability of wrongly rejecting a valid null hypothesis). Strictly speaking, this only applies to the primary endpoint of a study, while other endpoints no longer control the α-error at this level of significance. However, these endpoints are also used in the benefit assessment without critical discussion.
For the medicinal products covered by the GSAV, it can be assumed that the number of patients available for data collection during use is limited. At the same time, from a regulatory perspective, it seems sensible to limit the duration of data generation in order to be able to carry out the benefit assessment based on this within a reasonable period of time. Thus, the number of cases and the duration of the study are the two most important parameters that are usually used when planning a study in order to be able to derive a statement with an error level of 5%. Against this background, it is advisable - depending on the expected effect size - to adjust the required level of error according to the requirements of the individual case.
A comparable approach - albeit in relation to randomized studies - is also not foreign to the IQWiG: “In the case of very rare diseases, […] consideration could be given to allowing a larger statistical error level for regulatory decisions […] By increasing the two-sided error level to 10% and below Assuming a simple model (t-test), the number of cases could be reduced by a good 20% with a power of 80%, and by a good 40% with a two-sided error level of 20%. The advantage of such an approach would be to be able to at least quantify the possibility of error.” [17].
In situations in which proof of a patient-relevant effect does not appear possible within an acceptable period of time (e.g. due to low patient numbers), the EMA takes an alternative approach. The acceptance of surrogate endpoints makes it possible to make statements about the benefit of a new therapy option in a shorter time, even with small patient numbers. The acceptance of lower requirements for the validity of surrogate endpoints is accompanied by a lower certainty of results, but could, however, be included in the discussion against the background of the legal intention with regard to the drugs covered by the GSAV.

The need for a “dramatic effect” does not correspond to the intention of the legislature
The evaluation of medicines based on data collected during use builds on the decisions already made by the EMA in special approvals. The inclusion of these decisions enables the congruence between the benefit assessment and the findings of the approval authority, as required by the AM-NutzenV. Adhering to the requirement of a “dramatic effect”, which is customary in regular benefit assessment procedures, would not do justice to the legal intention and should be dropped with regard to the medicinal products covered by the GSAV.
The G-BA's criticism of indirect comparisons that were not carried out using a common bridging comparator often goes along the lines that, despite adjustment, the therapy effects to be compared are not of a magnitude that can rule out that the observed effects are due alone systematic bias or incidental findings.
However, the claim of wanting to rule out an incorrect assessment with absolute certainty runs counter to the assessment of the certainty of results and the probability of error, which is also carried out in regular assessment processes. Already the usual acceptance of evidence with a certainty of results of "hint" or "hint" implies a remaining uncertainty. Ultimately, even with a “proof” using one
95%igen confidence interval an error can never be ruled out. The explicit mandate of the legislature includes the acceptance of evidence with less significance than RCTs develop. Against this background, limiting the acceptance of a historical comparison to dramatic effects is not effective. It makes sense that the basic acceptance of historical comparisons should in future extend to the special features of the approval covered by the GSAV.
Regarding the requirements for the usability of historical comparisons, the G-BA states: "Due to the per se low certainty of results of a historical comparison, both completeness and a recording method that is as similar as possible of the underlying data, especially with regard to prognostic factors, representativeness and selection of the possible controls , central prerequisites” [18]. Examples of the recognition of historical comparisons can be found in the case law of the G-BA and confirm their basic suitability [19].
Recommendations for action for evidence generation and evaluation
The medicines covered by the legislation with “special” approvals have in common that the evidence from RCTs usually required in the benefit assessment process is not available or is only available to a limited extent and/or an additional benefit compared to previous therapy options has already been identified with the approval.
Based on these findings already made by the approval authority, the consideration of non-randomized data required by the GSAV to quantify an additional benefit enables the congruence between the benefit assessment and the findings of the approval authority required under the AM-NutzenV. The acceptance of the associated lower certainty of results compared to RCTs at the study level corresponds to the intention of the legislator and therefore does not fundamentally speak against the use of such studies. Possible sources of error must be kept as low as possible by using special methods. At the same time, the extent to which remaining distorting factors influence the study results must be estimated as best as possible. It cannot be assumed that the medicinal products covered by the new legislation can be assessed using a generic concept. Rather, a suitable approach will have to be specifically sought for different situations in order to generate the necessary evidence as far as possible.

Evidence generation
A two-stage process is recommended for this purpose, in which, in a first step, the evidence still to be expected is assessed with regard to its suitability for the benefit assessment process. In a second step, it is then important to identify the highest possible level of evidence and, based on the expected findings, to decide on application-related data collection and its design.
In principle, if an RCT exists or is expected, data collection should not be carried out during the application, as the benefit assessment can be carried out on the basis of a higher level of evidence. In this case, if questions regarding the benefit assessment remain unanswered, the time limit and reassessment can take place at the time the study results are submitted.
For the majority of the drugs covered by the GSAV, further evidence can be expected from the approval process even after market entry. CMAs are generally subject to requirements from the EMA, although these can be different types of studies.
It is recommended to seek dialogue with the approval authority at an early stage and strive to harmonize product-related requirements. In this way, e.g. B. Data from a European register can be used extensively, while a requirement of the G-BA could only be effectively implemented within the German healthcare context.
If no evidence can be expected that is suitable to quantify an additional benefit either entirely or after additional requirements, it must be examined at what level of evidence collected during use can be collected to quantify an additional benefit. The AM-NutzenV already provides for the consideration of study data with lower levels of evidence if it is “impossible or inappropriate to carry out or require studies with the highest level of evidence”. Prospective comparative cohort studies are planned below the RCT in the order of evidence levels to be used as defined there. Such studies could, for example, be carried out based on an indication-related register.
However, the sensible feasibility of such studies must be particularly examined for the drugs covered by the proposed law for two reasons.
On the one hand, the G-BA has the opportunity to ensure comprehensive implementation by linking data collection to the ability to prescribe only for active ingredient-related registers. Although disease-related registers may be required by the G-BA, the completeness of the register may not be guaranteed.
On the other hand, the drugs in question close existing gaps in care or offer significant benefits compared to existing therapy options. Against this background, a significant selection bias would be expected in a prospective comparative cohort study. Patients who receive the new therapy and patients for whom no therapy options or the previously available therapy options are used, e.g. B. differ in the severity of the disease. Only if this selection bias appears controllable can a meaningful comparison be made based on this study design.
If a prospective comparative cohort study cannot be meaningfully carried out, single-arm studies with a historical comparison represent the best possible study design.
The decisive factor for a reliable historical comparison is the availability of data that depicts the natural course of the disease or the course of the disease under standard of care conditions. For this purpose, a comparison with existing registers or a study research must be carried out before the imposition of the requirement.
Furthermore, it is necessary to be able to extract patient-specific data from the historical data, which can have prognostic relevance for the course of the disease. If these data are available to a sufficient extent, a similarity of characteristics between the intervention arm and the historical control arm can be established using a suitable adjustment procedure. The relevant patient-specific data must then be recorded as baseline data at the beginning of data collection accompanying the application.
At the same time, a comparable operationalization of the endpoints between the historical control group and the intervention group must be ensured.
It is also conceivable - using or recording the inclusion and exclusion criteria of the clinical study - to start a registry before approval is granted in order to achieve an ideal historical comparison arm. The prerequisite for this is an early decision as to whether data collection during use is planned for the drug in question. This approach is limited by the fact that, especially in the case of rare diseases, the majority of the patients in question are often already treated in clinical studies.
If a historical comparison cannot be meaningfully carried out, it should be examined whether the required evidence can be generated using a pre-post design.
Such a comparison makes sense if no spontaneous improvement can be expected for a (fictitious) untreated control group. If there is no reliable knowledge about the course of the disease without the new therapy option, an assessment should be obtained from specialist societies or specialized practitioners.
Taking into account the objectives of the GSAV, data collection must be able to contribute to improving the evidence base. Data collection can therefore only be viewed as justifiable if it can be expected that these data can fundamentally be used to quantify an additional benefit. If none of the above-mentioned requirements are met, data collection should not be required, as it is then generally not to be expected that the data collected can be used to quantify an additional benefit.
When selecting the most suitable form of evidence, existing EMA requirements must be given priority. The most suitable form of evidence should be identified under the requirements and, if necessary, supplemented with further aspects of data collection.

Evidence assessment and evaluation methods
For the medicinal products covered by the GSAV, it can be assumed that the number of patients available for data collection during use is limited. At the same time, it seems sensible from a regulatory perspective to limit the duration of the data generation that has been initiated, in order to be able to carry out the benefit assessment based on this within a reasonable period of time. Thus, with the number of cases and the duration of the study, the two most important parameters are already established, which are used in planning a study in order to be able to derive a statement with the usually accepted error level of 5%. Maintaining the requirement for a 95%igen confidence interval would therefore either thwart the legally intended possibility of demonstrating a quantifiable additional benefit in the case of limited patient numbers or would only allow this within an undesirably long period of time. Against this background, the required level of error should always be included in the consideration of suitable evaluation methods and - depending on the expected effect size - adjusted according to the requirements of the individual case.
With the consideration of non-randomized data provided for in the GSAV, randomization, which can ensure the similarity of characteristics between the study arms to be compared, is no longer necessary. However, there are several fundamentally suitable methods available with which sufficient similarity of features can be achieved by adjusting for different patient characteristics. Common and internationally recognized procedures include: B. propensity score matching (PSM) or matching-adjusted indirect comparison (MAIC). The basic prerequisite for such procedures is the knowledge and consideration of significant distorting factors. However, unlike randomization, these adjustment procedures cannot detect unknown confounding variables. This results in a lower certainty of results, which, however, is intended to be accepted with the legislator's specifications for the consideration of non-randomized data, which are based on the EMA's approval decisions for this special group of medicinal products. In principle, adequate adjustment in this situation enables valid statements about the additional benefit. No further requirements for the expected effect size are required. The size of the effect can increase the certainty of the statement to be made. However, the existence of a "dramatic effect" for the derivation of an additional benefit on the basis of non-randomised data as in regular benefit assessment procedures is not necessary and would not do justice to the legal intention of the GSAV data collection.
The selection of suitable adjustment procedures should be done prospectively. The selection of disruptive factors should also be determined with the involvement of scientific societies or specialized practitioners.
Leading international institutions of evidence-based medicine offer suitable instruments for evaluating the evidence collected during use. The GRADE approach represents an internationally accepted evaluation concept for bodies of evidence that upgrades or downgrades the underlying studies based on defined criteria and thus reliably assesses the quality of the evidence. The aim is to achieve low, moderate and even high levels of evidence for non-randomized data. What seems remarkable is the fact that the contribution of an upgraded observational study to the body of evidence can reach the same level of quality as an RCT. The instrument can therefore offer a suitable framework for the recognition of application-related data for benefit assessment.
With REQueST, a tool for consistently assessing the suitability of registries for HTA was developed as part of EUnetHTA. This instrument can be used to evaluate the quality of existing registers. At the same time, it presents criteria for a newly established, high-quality register in a comprehensible manner. Other instruments, such as ROBINS-I developed by the Cochrane Collaboration to assess biases in non-randomized studies, can be used additionally if necessary.

Specifications of the G-BA consultations
In accordance with the requirements of the GSAV, the G-BA must define requirements for the duration, type and scope of data collection and evaluation and in particular determine specifications for methodology as well as patient-relevant endpoints and their recording. In order to achieve targeted data collection accompanying the application, it should be ensured in this context that complex data collection is avoided, which is unlikely to contribute to the quantifiability of an additional benefit.
Regardless of the type of study chosen, the methodological principles of analyzing the data should be discussed in advance between the G-BA and the pharmaceutical company in view of the different conceivable evaluation options. The essential aspects of data generation and evaluation, including the question (PICO scheme), form of evidence, confounding factors and adjustment methods, sample size, duration and acceptable statistical certainty should be determined in advance.
These consultations should take place as early as possible through dialogue with the EMA and should be binding in nature. At the same time, the early dialogue with the approval authority should be used to strive for harmonization of the conditions of approval and G-BA. For the medical assessment of the feasibility of the requirements, the medical specialist societies should also be involved and specifications should be made in agreement with the approval authorities. <<

Do you want to leave us already?

Subscribe to our newsletter quickly. Then you can be sure that you won't miss anything.