Surrogate parameters and patient-relevant endpoints: fallacies and consequences

In Germany, medical care is largely financed by contributions from insured persons. This results in the normative and ethical obligation of statutory health insurance to provide the optimum level of health services with maximum benefit for the insured. It is important to reduce waste, use resources efficiently, and all results must be meaningful, especially for patients. The health insurance companies, together with the service providers and the patients, are responsible for achieving these goals and a primarily scientifically based approach (SGB V §§ 2, 12, 35b, 70). The question of what is useful is described, for example, in SGB V § 35b as “improving the state of health, shortening the duration of the illness, extending the lifespan, reducing side effects and improving the quality of life”. This maxim can be found in an identical form in the G-BA's Rules of Procedure, in the Drug Benefit Assessment Ordinance or in the IQWiG method paper for benefit assessment. The statement that the benefit assessment has to be based on “patient-relevant target values” is of particular importance. “In this context, “patient-relevant” should be understood as how a patient feels, can perceive their functions and activities or whether they survive.” (IQWiG 2011, p. 30f)

>> These definitions also outline the goals of new treatment procedures and the parameters of a benefit assessment. The target criteria are often not easy or quick to collect, but require scientifically reliable evidence. The survey can be expensive - it requires studies with complex designs. In addition, the measurement of patient-relevant endpoints requires time that is (apparently) not available when there is an urgent need for action in diseases with high mortality without sufficient treatment options.
To solve these problems and quickly evaluate the results of medical treatment, surrogate parameters are used. These surrogates serve as substitute parameters for the actual clinical goals. They should be collected earlier and easier and should adequately represent the actual target variables.
What all surrogates have in common (Mangiapane, Garrido 2009: 14) is that they usually represent precursors of the actual clinical endpoint that are not immediately perceivable by the patient. Biomarkers are often used for this, such as glycosylated hemoglobin (HbA1c) or LDL cholesterol. The correlation between surrogate and clinical endpoint (individual-level association, Buyse 2009) as well as the causal connection between treatment effects on surrogates and effects on clinical endpoints (trial level association, ibid.) are explained biologically or pathophysiologically. Surrogates are considered valid if association and causality have been reliably proven. This validation of surrogates in each of their areas of application is an independent, indispensable task that “…requires extensive research, including the conduct of randomized controlled trials (RCTs) with clinically relevant endpoints” (Mangiapane, Garrido 2009: 1).
The experience with surrogate endpoints
An example of a sensible use of a surrogate parameter is the assessment of the effectiveness of antiviral therapy (AIDS/HIV) based on the number of CD-4 cells and the detection of virus RNA (Glaeske 2010: 162). These parameters correlate with the progression of the disease and provided the basis for the drug's approval by the American Food and Drug Administration. This made rapid approval possible, which saved the lives of many people with AIDS and was therefore more effective than waiting for opportunistic infections or deaths to occur.
However, experiences with surrogate endpoints urge caution and extreme skepticism. Too often in the past, supposedly successful interventions on surrogates have, over time, after further investigation, turned out to be not only useless, but unfortunately often also harmful.
The use of surrogates has therefore come under increasing criticism in recent years. Studies that only report surrogates can hardly be scientifically justified. This is clearly demonstrated by the so-called surrogate fallacies (Mühlhauser, Berger 1996; Mühlhauser, Meyer 2006). This is when medical interventions have desired effects on the surrogate, but no or even opposite effects on the endpoints relevant to the patient. Examples of this can be found in Table 1.
The HTA report “Surrogate endpoints as parameters of benefit assessment” (Mangiapane, Garrido 2009) shows that surrogates are almost not used in Health Technology Assessment (HTA) reports to assess the relevance of medical measures worldwide and their use is also not recommended .
These bad experiences with measuring medical effects on the basis of surrogates and the above-mentioned obligation to a primarily scientifically based, harm-minimized treatment will require in the future to largely avoid using surrogates and to exclusively evaluate medical procedures on the basis of patient-relevant endpoints (Glaeske 2011).
The reality
Nevertheless, groundbreaking decisions in healthcare are often made based on surrogates. The study by Ujeyl et al (2012) on the availability of data when drugs are brought onto the market shows that when drugs were approved, a primary patient-relevant endpoint was only recorded in 46% of the cases.
The Joint Federal Committee (G-BA) decides which interventions will be taken over by the GKV. When making these decisions, he is also committed to the principles of evidence-based medicine, according to which new methods are only to be evaluated based on “patient-relevant effects” (G-BA method evaluation). In fact, however, this principle has been undermined again and again: With a resolution of July 15, 2007, skin cancer screening from the age of 35 was decided to be a statutory health insurance service (BAnz. No. 37, p. 871) of March 6, 2008). The underlying reasons for the decision make it clear that there is no evidence for patient-relevant outcomes such as the reduction in melanoma-related mortality. The decision is justified with the surrogate “advancing the time of diagnosis” and the possibility of treating early stages of melanoma curatively (G-BA 2007). Early detection is an offer for the healthy population as a whole that can be used by more people than a measure for a specific indication. The G-BA has made a decision here that has significant financial consequences and unclear benefits for patients based on surrogate endpoints.
With the decision of the G-BA and the requirements of certification commissions to prescribe minimum quantities for certain procedures in order to ensure the quality of results, surrogates are also finding their way into quality assurance. There is no data that proves a causal relationship between quantity and quality of results (Weißbach/Schaefer 2010). Even if, according to the Federal Social Court (judgment of September 12th and December 18th, 1012), this is not necessary and the setting of minimum quantities is generally permissible from a legal point of view, the quantity remains a surrogate for the quality of results, which according to the common definition (see introduction) is not is a valid parameter because there is no evidence of a causal relationship. This is extremely worrying in view of the incentives that arise primarily from the certification requirements. The “2012/2013 transplant scandal” was not the only thing that showed what effects the uncritical handling of an invalid surrogate can have.
Consequences
As shown at the beginning, it is theoretically well established that surrogates are subject to uncertainty and empirical evidence shows that in many cases their use has led to false conclusions regarding patient-relevant endpoints. They are often used in studies because only by choosing the (surrogate) target parameters can a significant (positive) effect of the intervention be proven. The result: knowledge about the best – or at least the appropriate – care is distorted, among other things, by the use of surrogates, and patients consequently receive a less useful or a more harmful intervention. This practice, which is devastating to the care and well-being of patients, must not be accepted but must be ended. Consequences need to be drawn: expansion of the scientific basis for decisions in the healthcare system and the consistent participation of patients.

The “right” studies
The first consequence of the uncertainties is the demand for the “right” studies. In the Drug Benefit Assessment Ordinance of December 28, 2010, the legislature clearly defined the benefits of drugs only in relation to patient-relevant endpoints. It therefore seems logical and necessary to also use this definition for approval. It would therefore be clear to demand that the parameters that are supposed to apply to assessing the benefits according to the law are also mandatory for approval. Registration studies must primarily test the patient-relevant endpoints of morbidity, mortality and quality of life. Others should not be accepted. And analogous to the initiative of the International Committee of Medical Journal Editors to only accept prospectively registered studies for publication in order to avoid publication bias (ICMJE recommendations, 2010), publication of studies could also be linked to the obligation to report patient-relevant endpoints. The CONSORT statement (the CONSORT Group 2010), which formulates standards for reporting the results of clinical studies, should also clearly demand their consideration and define when surrogates can be permitted. Perhaps the US Patient-Centered Outcomes Research Institute (www.pcori.org), in which many high-ranking representatives of the American healthcare system are involved, will provide important impetus to anchor patient-relevant endpoints as good practice for all clinical studies.

Patient involvement
A second, crucial consequence is the demand for comprehensive patient participation at all levels of the health system - including in the planning and evaluation of health research studies. Patients are directly affected by the possible damage caused by incorrect system decisions. The more their perspective is integrated into the decision-making process, the greater the chance that decisions will be made based on criteria that are actually relevant to them. However, the prerequisite for this is that patient participation is taken seriously, implemented according to a defined methodology (e.g. a transparent nomination process) and the representatives are trained accordingly (Berger 2010). A distinction must first be made between participation at the system level - i.e. in decision-making committees, in knowledge generation and transfer - on the one hand, and at the individual level - i.e. in decision-making in partnership - on the other hand.

System level
Today, patients are not only involved in decisions at the health policy level, their participation is even legally mandatory (SGB V §140f, g) and the procedure is legally regulated in the “Patient Participation Ordinance” (PatBeteiligungsV, BGBl. I p. 2753). Certain patient organizations have the right to send representatives to the G-BA, who have the right to be heard in all decisions, but not to vote.
When transferring knowledge, for example when creating medical guidelines or patient information, the involvement of patients is not mandatory, but is defined as a quality criterion - nationally and internationally (DELBI, AGREE, GRADE, GIN Public Toolkit, GPGI). They have full voting rights in guideline groups (ÄZQ 2008), but they are currently not even involved in half of all high-quality guidelines (development level/S 3) across Germany (Schaefer 2010).
Patients have so far only been included sporadically in the development of HTA reports and systematic reviews (Kreis 2012, Facey 2010); there is a lot of catching up to do in the design and implementation of clinical studies. Although the demand for patient participation is particularly vehement in this area (European Science Foundation 2012), there are so far only a few corresponding initiatives (e.g. the James Lind Alliance in Great Britain). The patient perspective would be essential here in particular in order to ensure that patient-relevant endpoints are taken into account in the design of clinical studies and to avoid invalid surrogates as far as possible.
In summary, there is a consensus that patient participation at the system level is desirable because it ensures the consideration of patient-relevant endpoints and thus the relevance of system policy decisions. However, it is rarely practiced.

Participatory decision making (PEF)
At the level of the individual doctor-patient relationship, it is now generally taken for granted that patients are included in the decision-making process (Härter et al 2011). Ideally, they can then weigh up the benefits and harms of therapeutic or diagnostic interventions against needs, values and life situation in order to make a “right” decision for them. To do this, they must know and understand the available and reliable data on the benefits and harms of the decision alternatives (Klemperer et al 2009). So you need understandable, systematically researched and high-quality information. This information must report endpoints that are important in their decision-making situation. However, a benefit suggested based on invalid surrogates can lead to a decision that is harmful to health. This is how e.g. For example, in the early detection of ovarian cancer, improved five-year survival after transvaginal ultrasound (TVUS) is communicated as evidence of benefit (van Nagell 2011). However, the improved five-year survival results solely from the advance of the time of diagnosis and is not in itself a valid parameter for the benefit of early detection examinations (Wegwarth/Gigerenzer 2011). In fact, not a single death is prevented as a result of TVUS, but many additional oophorectomies are carried out as a result of overdiagnosis, which harms patients without influencing the course of the disease (Buys 2011).
The new law to improve the rights of patients (Patient Rights Act - PatRG 2013) emphasizes the importance of education and information for patients. This accentuation becomes clear, for example, in the clear distinction between security education (BGB § 630c) for participation and averting danger and self-determination education (§ 630e) for information about the course, risks and alternatives.
So far, however, only rough information on benefits and side effects has been legally required for the doctor; both do not have to be quantified: according to general case law, the doctor only has to inform the patient “in general” about the opportunities and risks of a treatment (BGHZ 90, 103ff ). The special report of the Expert Council 2012 also emphasizes the need for high-quality patient information and calls for a central institution for this task. However, the basics of information (e.g. valid patient-relevant endpoints, high-quality studies) are neither addressed in case law nor in the Patient Rights Act or in the current report by the Expert Council for the Assessment of Developments in the Health Care System (2012).
At the system level, patients involved can request that surrogates not be reported or that no recommendations for action be made or system decisions made on their basis. At the individual level, patients can only pay attention to whether the benefits and harms of the options for their current decision have been examined using hard endpoints and whether these have been reported in an understandable manner. And they can decide whether these endpoints are also relevant in their personal situation or for themselves.
Current challenges
Implementing the two previous calls for meaningful studies and more patient participation can help ensure that many system and treatment decisions are no longer made on the basis of surrogates and are therefore susceptible to false conclusions. Specifically, some current developments in the healthcare system appear in a new light if they are analyzed exclusively for their benefit with regard to patient-relevant endpoints and invalid surrogates are declared inadmissible in the benefit assessment.

Personalized medicine
The so-called “personalized medicine” is a focus of innovation policy and health research (BT-D 2010, Federal Government Health Research Framework Program, Hüsing et al. 2008). She is pushing powerfully into the health policy discussion (German Ethics Council 2012; Schumpelik, Vogel 2011; Müller 2012 on the relevance of the concept for the GKV). The term mixes two concepts:
On the one hand, this brings with it the promise of being able to offer each patient treatment tailored to their biological and genetic disposition: specific active ingredients are developed for certain genetic or biological characteristics. By limiting the spectrum of effectiveness of pharmaceuticals to a narrow group of patients, a common disease turns into individual, rarer diseases (orphanization). This makes the requirements for a solid evidence base more difficult. At the same time, high-priced drugs are pushing their way onto the market; the substances are often to be approved in an accelerated process and on the basis of interim results and surrogate endpoints.
This can be seen, for example, in a “blockbuster” of personalized medicine: Bevacizumab, an angiogenesis inhibitor from Roche, was approved in an accelerated procedure in 2008 for the treatment of metastatic breast cancer. The basis for approval was a significant reduction in progression-free survival in an interim analysis, which was accepted by the approval authorities as a surrogate for the reduction in overall survival. However, in November 2011, the American FDA withdrew the drug's approval for this indication: the analysis of the data now available showed no improvement in overall survival despite a longer progression-free interval (FDA November 18, 2011). The current Cochrane review comes to the same conclusion (Wagner 2012). However, the European Medicines Agency (EMA) sees no reason to withdraw the approval. For them, progression-free survival as a surrogate is completely sufficient (EMA 2012). The enormous financial impact the FDA decision had can be seen from the publications in leading business magazines: Roche's top-selling drug collapsed, as did its share price. In the meantime, sales are being driven up with a highly dubious “pay-for-performance” strategy: If bevacizumab does not work, Roche reimburses the hospital for the treatment costs, thereby creating an incentive to harm the patients. However, what is relevant for patients doesn't count: what came as a promise has long since become an aggressively expanding market. Surrogates are one of several means of providing market access for medications that are often ambiguous in terms of their benefits and harms.
Secondly, “personalized medicine” is associated with a prediction concept according to which disease risks and treatment chances of individual people are assessed exclusively based on their biological characteristics (e.g. genetic, cytological, morphological information). Medical treatment as well as the lack of treatment are determined and legitimized by operationalizable biological characteristics of the individual. In this form of prediction, the surrogate parameter becomes a principle: Bio- and genetic markers are surrogates in the classic sense: There is a retrospectively proven connection between the marker and the frequency of certain diseases or a certain course of the disease. However, these surrogates are not valid in terms of the methodology: There is no evidence of causality, i.e. that influencing the marker also influences morbidity, mortality or quality of life. This is only possible with randomized controlled trials that test for these hard, patient-relevant endpoints (Buyse 2009, Fleming & Powers 2012). The further forward the time of diagnosis is moved and pre-diseases are diagnosed, the more difficult it is to prove a connection between the surrogates and patient-relevant outcomes in a randomized study design: The possible variances in further diagnostics and treatment become larger and so do the results less and less comparable. Prediction in the sense of “personalized medicine” is therefore based exclusively on non-validated surrogates. Whether this prediction has an actual benefit in terms of disease burden, mortality or quality of life is unlikely to be proven. The damage, on the other hand, is obvious: instead of illnesses, risks of illness are diagnosed, although their actual occurrence is uncertain and their value for the patient is often highly questionable. This means that many people become unnecessary patients (and possibly treated unnecessarily).

Prevention
A similar problem is also easing in the area of prevention. The belief has become increasingly widespread that it is better to prevent illnesses than to treat them later at great expense. This gives rise to many prevention initiatives, such as regular health “check-ups”, campaigns for a healthy lifestyle, and early detection measures, such as those intended to be implemented with the Cancer Plan Implementation Act (draft bill 07/2012). The draft bill for a law to strengthen health promotion and prevention (draft bill "Prevention Law" 2013, as of January 11, 2013) clearly formulates the goal of avoiding illness, promoting people's health and reducing the costs of illness and the consequences of illness by strengthening preventive measures. The draft bill also mentions “ensuring the quality and promoting the effectiveness of prevention and health promotion” as a measure to achieve the goal - without, however, providing for specific regulations in the law on this point and without strengthening the evidence-based basis of preventive measure recommendations at all. This is unusual in that a law usually contributes to the solutions sought by the law and evidence-based proof of effectiveness and ensuring the quality of preventive measures represent a challenge.
The prevention goals are honorable, but previous assumptions about the benefits of prevention are largely based on surrogates. There is little data on patient-relevant outcomes, but it is thought-provoking: A current Cochrane review shows that so-called general health checks (check-ups) have no effect on the morbidity and mortality of the population, but rather significantly increase the overall costs for diagnostics and treatment (Krogsboll 2012). This is diametrically opposed to the Prevention Act's requirement to offer check-ups more frequently and earlier.
For early cancer detection, it is now being discussed whether proof of a reduction in tumor-specific mortality is sufficient as a benefit (Schaefer et al 2011). Proving a reduction in overall mortality caused by early cancer detection is almost impossible for statistical reasons; this would require studies with millions of participants, which are probably not feasible purely logistically (Dubben 2009). It will also not be possible to validly determine the effect of screening on quality of life: A problem that arises in long-term studies on quality of life is the phenomenon of “response shift”: In general, every person adapts to new situations that determine them allow events to be judged differently. (This phenomenon was noticed in the early phase of the EORTC QoL group, as even when quality of life was measured in more than 10,000 cancer patients, hardly any therapy-dependent or stage-specific changes were detectable.) The legally required patient-relevant endpoints will therefore in principle not be verifiable for early detection measures.
Last but not least, it must also be asked whether the assumptions about the benefits of primary prevention meet the requirements for valid endpoints. Prevention focuses on general health maintenance. So far, however, only certain indication-related endpoints have been examined, such as the frequency of malignant neoplasms, respiratory diseases and cardiovascular events in the case of smoking. These are accepted as surrogates for general health. For none of these endpoints has it been proven so far using the methodology described above in a randomized controlled design that influencing them causally improves (overall!) mortality, morbidity or quality of life. Ultimately, our assumptions about the benefits of primary prevention are based on unvalidated surrogates. That doesn't have to speak against prevention. It should only encourage people not to overestimate the benefits of prevention and not to view maintaining health as an obligation. “In order to become demented, I first have to survive the heart attack,” a health politician once said. Only they can decide which of the two forecasts is more relevant for a person based on their personal experiences and values.
Conclusion
Surrogates can only be considered reliable endpoints if they have been validated. The causality to relevant clinical endpoints necessary for this validation has so far been proven for very few surrogates. On the contrary, interventions that achieved desired changes in surrogate parameters were later observed to have a different, detrimental effect on clinical endpoints. Studies that only report surrogates therefore do not provide reliable results. Nevertheless, in the reality of care, important decisions are often made based on surrogates. It is necessary to finally draw the resulting consequences: to finally anchor the requirement for studies with clinically relevant endpoints in all regulations and to consistently involve patients at all decision-making levels in healthcare. Above all, it is urgently necessary to examine upcoming developments and decisions in the healthcare system to see whether the assumptions underlying them are based on valid patient-relevant endpoints. <<