“Stronger focus on the methods of study conduct”

Following Prof. Dr. med. Peter Sawicki (2004 to 2009) and Prof. Dr. med. Jürgen Windeler (2010 to March 2023), Dr. med. Thomas Kaiser has been the third Director of the Institute for Quality and Efficiency in Health Care (IQWiG) since April 2023. In an interview with "Monitor Gesundheitsforschung," Kaiser, a founding member of the staff of the "non-profit foundation for quality and efficiency in health care," outlines the goals for the coming years. A committed EbM expert, he aims to strengthen international connections and participation in EU projects and, as he puts it, to devote himself to "more scientific investment beyond the processing of our mandates."

>> I remember well that the early years of IQWiG were shaped by Prof. Dr. Peter Sawicki, a diabetologist and previously co-editor of the pharmaceutical-critical Arznei-Telegramm: the term "eminence-based" was making the rounds. He was followed in September 2010 by Prof. Dr. Jürgen Windeler, previously head of the Evidence-Based Medicine Division of the Medical Service of the Federal Association of German Physicians, and with him the discussion on efficiency and effectiveness, as well as the quality and usefulness of registry data. What's in store for you, Dr. Kaiser, who has headed IQWiG since April 1, 2023, almost exactly two years ago?
I would love it if one day, on behalf of IQWiG, I could embrace the term "international institution" on the one hand, and the term "providing constructive support for better studies" on the other. The first is almost automatic, because IQWiG must become an international institution simply because the EU has established the appropriate framework for this. Since January of this year, we have had the early benefit assessment of medicinal products at the European level, or EU-HTA for short, which alone will lead to a greater internationalization of the institution. Medical devices will follow.

Their approvals are granted at the European level. The Federal Institute for Drugs and Medical Devices (BfArM) is responsible for this for Germany.
Correct, but only for approval. At the European level, a comparative benefit assessment will be introduced for medicinal products and individual medical devices – similar to the AMNOG (German Medicines Act). The vast majority of this will be undertaken by the IQWiG in Germany, and a small part by the Federal Joint Committee. To this end, collaborations will be established with other European HTA institutions, European committees will be appointed, and of course, all of this will be conducted in English. For IQWiG, tasks in the pharmaceutical field will be primarily carried out by Beate Wieseler and her team, the chair of the Methods Group at the European level. We will also use this opportunity to develop and publish international publications if they serve IQWiG's objectives.

Do you want to create more of your own publications?
Exactly. During my term in office, I would like to further advance the institute as a whole. One will be stronger international ties and participation in EU projects, which has only happened sporadically so far; the other is a shift toward greater scientific investment beyond the processing of our mandates. Our main focus will, of course, continue to be on processing the mandates we have been given—especially benefit assessments—but we will create opportunities and freedom to also conduct scientific work. This, by the way, is not art for art's sake; it is essential if we want to do the commissions we face as a European institution not just well, but excellently. To this end, it is important to also develop scientifically.

In which area, for example?
For example, we not only want to further develop the AMNOG procedure – as we have already done – but also, together with the evidence-based medicine community, place a stronger focus on study conduction methods. The further development of method development has so far mainly focused on how studies are evaluated and how systematic reviews can be conducted. But now, beyond the purely procedural issue, we have begun to pay more attention to how study conduction methods can be further developed and improved. Our goal is not only to produce more good and meaningful studies more efficiently and reliably, but above all to produce feasible ones. This is a crucial point that must never be overlooked: studies must be feasible. This is especially true in the area of very rare diseases, where the evidence base is relatively often insufficient. This is certainly partly due to logistical difficulties, but is also due to the fact that existing study designs and options for statistical data collection and analysis are either not being used or have not yet been fully researched and developed.

In an interview with "Ersatzkasse Magazin" (1), you stated that you would like the IQWiG to become more involved in the future and to contribute its expertise more intensively to the design of studies, i.e., to generate the necessary evidence. How will this be achieved beyond the consultations already offered?
We have had various points of contact for some time, for example, through application-related data collection. Here, we participate in the deliberations of the Federal Joint Committee, drafting specific study concepts and reviews of study protocols and statistical analysis plans on its behalf. We also have experience in outlining and discussing pilot studies in the field of method evaluation.

However, it is not (yet) the mandate of the institute to conduct studies itself.
We don't expect to receive such a permit, nor is that the goal of IQWiG. However, our goal is to contribute our expertise in evidence generation even more strongly within the regulatory context of the Federal Joint Committee (G-BA). Ultimately, everyone benefits from this: patients, physicians, and trial managers.

Where exactly do you want to contribute your expertise more?
These include questions such as: How can better studies be conducted? What are good recruitment strategies? What options are there for working with specific endpoints? How can missing values be avoided or dealt with? How can patient-reported endpoints be routinely included? These are all important topics where we want to engage scientifically at a general level and thus provide assistance. This also benefits us and the G-BA: The better the studies and their results, the easier and more valuable our work will be in the future.

Knowing that there are many groups that already have very good experiences with conducting studies.
True. But we have a different perspective when analyzing studies. We have a wealth of expertise and experience to say quite precisely why the results presented to us aren't as good as they could be, if perhaps just a few small details had been changed beforehand. You can learn very well ex post what you could have done better at the beginning. And we should use that instead of constantly moaning that a study simply didn't go well.

The pharmaceutical industry will surely be grateful. I don't know of any pharmaceutical industry manager who would want to waste money launching studies whose results are subsequently not recognized or only partially recognized because, for example, the endpoints were not chosen correctly. But wouldn't that be more the responsibility of clinical research or health services research?
Certainly, although they may not yet be fully aware of the problems we face. But this could also be a question of the general research culture, which always—which is entirely understandable—seeks to explore new things instead of creating a cycle of improvement through ongoing research. That's precisely what drives us.

IQWiG is currently involved in the EU project RealiseD (2) with the aim of developing clinical trials for good evidence on very rare diseases in order to arrive at effective therapies more quickly.
The EU project RealiseD is explicitly aimed at researching how to conduct better comparative studies on very rare diseases.
From which we can certainly learn for other indications as well.
Certainly. It will be possible to draw conclusions, although we are primarily concerned with truly rare diseases. These specifically involve questions of data collection and specific statistical methods that can be used to generate new insights without too significant a loss of reliability.

Where do you see your role as an HTA institution?
We no longer want to conduct mere post-hoc evaluations, but rather help answer the question from the outset as to whether a new drug for a very rare disease offers an advantage over existing treatment options. This requires appropriate tools, some of which have yet to be developed, so that the questions that arise during the final benefit assessment can be answered not with just any studies, but with the best possible studies. Therefore, I think it's fantastic that we've succeeded in becoming part of this important project, in which several European regulatory authorities, many scientific organizations, and the pharmaceutical industry are participating.

The IQWiG in a project together with pharmaceutical companies would probably not have existed under Sawicki and Windeler.
I don't want to judge whether this wouldn't have been possible under the leadership of my two predecessors. The situation back then was completely different than it is today.

You were there when the institute was founded.
Exactly. Let's go back briefly to 2004, when the Institute for Quality and Efficiency in Health Care was founded as part of the implementation of the Statutory Health Insurance Modernization Act (GKV-Modernisierungsgesetz) to improve the quality and efficiency of patient care in Germany. Until then, for example, the pharmaceutical industry could charge high prices for new drugs without having to prove that they were better than the existing standard of care. When that changed with the IQWiG, it came as a shock to some. Also because some may have assumed we were working against the pharmaceutical industry. But that's not true, because in my opinion the goal was – and still is – to make the best possible decisions for the benefit of patients based on the best possible data.
As always, this is a question of communication, according to the old German saying: What goes around comes around. Certainly, some things were handled incorrectly on both sides at the time.
I don't want to rule that out either. There are only people working on both sides. HTA organizations and the pharmaceutical industry play different roles on the same playing field; that cannot and should not be ignored. But we are not opponents. The pharmaceutical industry plays an important role in drug development, and without the drugs researched in this way, people would be much worse off than they are today. I am well aware that the pharmaceutical industry is not solely responsible for drug research. There are many good examples of how the development of drugs has improved people's lives, and studies were conducted for their approval that delivered truly good and meaningful results. On the other hand, there are also examples of studies that simply have not adequately answered the questions for patient care. And that is also why we have quite a few drugs in our healthcare system whose significance for healthcare we can hardly assess due to a lack of data. Our task is to point out this gap. But now we also want to help consider how studies and thus the evidence base can be improved. This is the path into the future that I want to take with the institute. I'm aware that this won't be an easy path. Because it's always easier to criticize than to improve.

Doesn't this also include a broader understanding of endpoints?
If there is good evidence that other endpoints indicate what one actually wants to see in order to clearly describe the benefit, then these are suitable endpoints. It goes without saying that these endpoints must be validated. Therefore, at IQWiG, we have a strong interest in creating the best possible data and research infrastructures to be able to answer precisely such questions. This is not just about improving individual studies, but also, for example, about defining and validating good intermediate endpoints that can be used in the future to determine even earlier whether a drug or therapy provides any benefit or none at all.

You just mentioned a research infrastructure: doesn't that also mean a closer integration of the institute with university institutions?
If that contributes to improving the current situation, absolutely. First and foremost, we will work on ensuring better mutual understanding in general. This also includes better understanding what each other is doing and what limitations exist in the actions of institutes like ours and on the part of the universities. I also see this as a personal task for me: to establish as many contacts as possible with important institutions, universities, and their chairs.

How exactly has healthcare improved in these more than 20 years of IQWiG's existence? Or should we view the whole thing as a step-by-step innovation?
If I had to give an evidence-based answer, I would have a very difficult time because that would require a controlled comparison. One would have had to define a zero point, so to speak, in order to determine what has happened in terms of evidence-based research over the past two decades, with and without IQWiG. Of course, one can say that there have been blatant advances in this time, for example in the treatment of cancer. But that is not our merit; rather, it is the merit of drug research. Our merit is certainly that we contribute to ensuring that these active ingredients are used specifically precisely where there has actually been significant progress – and are not used, or at least used more cautiously, where that is not the case. In addition, the institute has been responsible from the outset for developing and making generally understandable information available. This ensures that everyone directly involved in treatment can make evidence-based decisions about healthcare services.

Speaking of transparency: The IQWiG has a legal mandate to provide health information to patients. Where do you stand on this? And are you succeeding in reaching people?
On our website www.gesundheitsinformation.de we provide information about the diagnosis and treatment of the 300 most common diseases in Germany.

Do patients take advantage of this offer?
Of course. The IQWiG subsidiary website www.gesundheitsinformation.de had 3.7 million visitors in January 2025, who gathered information on topics such as shared decision-making, as well as on various diseases and active ingredients.
What about those population groups that are less likely to demand health information?
Unfortunately, our website only reaches those who actively search for health information online. We need to do a lot better in the future to also reach those target groups who are more frequently affected by health problems but belong to population groups that are generally considered to be less educated. We have already carried out projects to find out how such groups can better access important health information and are currently taking the next step in doing this with so-called intermediaries – organizations, clubs, associations, etc. We are also in the process of developing health information specifically for people who find written information more difficult. This could be videos, short podcasts or direct conversations with peer groups, for which we then provide the evidence-based information in various languages.

So far, this information is only available in German and English.
That will, and must, change. The use of artificial intelligence will certainly be a great help in this regard, although a cultural context must always be taken into account, which requires people who are familiar with different cultures.

You studied systems engineering (1990) and also received your doctorate in human medicine (1999) and your doctorate (2005). You co-founded the Institute for Evidence-Based Medicine in 2002, received the Bill Silverman Award from the Cochrane Collaboration in 2008, and received the David Sackett Award from the German Network for Evidence-Based Medicine in 2011. Which of the skills and experiences you have acquired most contribute to your daily work?
Actually, all of them. When we talk about research and data infrastructures, for example, it helps to have an understanding of data sets and structures. I also have a strong affinity for AI thanks to my profession as a systems developer. Incidentally, this also applies to my private life: at home, I make art with AI and set up the necessary servers myself. Shortly after I took office, we founded an internal AI working group at the institute. Its basic goal is to familiarize all employees with the topic of AI in general and to develop which AI applications can generally help us in our daily work – such as translations, improving and automating workflows with AI or literature screening, selecting studies and extracting data, etc. Topics also include AI in the interpretation of X-ray images, assessments based on medical histories, and triage.

The areas of application of AI are extremely diverse.
So countless and diverse that we've even established an AI playground. Here, we give our employees the opportunity to learn and see what AI can and can't yet do. AI will definitely change our world of work and assessment. That's foreseeable.

To what extent?
We'll see. But we need to stay on the ball to be able to recognize as quickly as possible when to best utilize which of the ever-increasingly fast-developing AI innovations to become even better and more efficient.

In response to another question from "Ersatzkasse Magazin" about whether you see any developments in the studies presented to you, you replied: "We can observe some positive developments, for example, in the endpoints reported by patients themselves. Quality of life and symptoms are examined more frequently today than was the case at the beginning of AMNOG." What is the general status regarding the inclusion of PROM and PREM in AMNOG?
PREM, i.e., whether someone was satisfied with something more or less, has not proven to be a viable approach in the context of the Social Code Book V. The situation is different with patient-reported symptoms, because it's always about determining how people are really feeling.

These are questions like: What symptoms and side effects do they have? How are these individually assessed and weighted?
Exactly. If, for example, you neglect the impact of diarrhea as a side effect of drug therapy on quality of life, you're ignoring an important consequence for patients. However, this side effect must be weighed against the desired change, for example, whether a large tumor has shrunk slightly. The decisive factor is not just whether the tumor has shrunk, but how the patient is doing. If I don't collect PROM, I'll always have a blind spot regarding the patient's experience.

Are PROMs collected consistently and in a structured manner?
PROMs traditionally play a subordinate role in drug approval and evaluation, as they do not require collection for approval. There are, of course, exceptions, such as in pain therapy, where pain reduction, measured as PROMs, is the primary outcome measure. Nevertheless, we are seeing some pharmaceutical companies increasingly collecting PROMs. This is not because they need the data collected in this way for approval, but because they are relevant to healthcare issues.

What about quality of life? One might assume that, as the EU HTA begins, the inclusion of the quality of life dimension will increase, as other countries are already much further ahead than we are in this area.
In fact, some countries aren't much further ahead than us in this regard, as we're pretty much a pioneer in this area. This has been an important topic since the founding of IQWiG, and we continue to place great emphasis on patient-reported outcomes.

Finally, let's look at a detail from AMNOG: the benefit assessment of the drug pembrolizumab (3): How is it that the IQWiG sees a lower benefit in one assessment, while the ESMO (4), a European medical society, sees the highest possible benefit? One might assume: evidence is evidence. Or does it depend on how you look at it?
The difference is quite obvious. In the ESMO assessment, only the observed benefit regarding recurrences plays a role. IQWiG also described the drug's benefit for recurrences, but weighted it differently than the ESMO. This is because the achieved recurrences are offset by a high proportion of severe and very severe side effects. Furthermore, the study presented showed that there was no improvement in quality of life, despite a reduction in recurrences.

So the fact that serious side effects are caused did not play a role in the ESMO assessment?
Exactly. That's precisely what we consider inadequate, to put it mildly. That explains the difference between the two assessments. On the one hand, you're right: evidence is evidence. On the other hand, it does make sense not to just look at 30 percent of the evidence, but to always consider 100 percent of the evidence when it's available. Unfortunately, a balancing of the pros and cons isn't visible in the ESMO assessment, even though some side effects were life-threatening or even required hospitalization.

We'll ask ESMO why this is the case. Finally, if you had one wish, and there's a new federal minister sometime in March or April, what would you wish for the IQWiG?
That they contribute as much as possible to promoting research in Germany. This includes recognizing the general value of research, supporting existing research infrastructures and building new ones, but also asking important research questions themselves.

For example?
Let's take the discussion about the treatment of people with long COVID. There are currently no approved drugs for the treatment of the diverse symptoms, only off-label therapies. Of course, it's possible to simply allow off-label use. However, it would be more sensible to combine this with reliable evidence generation, as the NHS in the UK has done, in order to learn for future patients and continually improve treatment options.

When did the British do this?
This happened at the very beginning of the coronavirus pandemic. Even back then, the UK decided to pragmatically launch a randomized, care-based, interventional study in a hospital setting—a first-class example of health services research, by the way. I would like to see something like this, or at least something similar, from the future Minister of Health: not trying to know everything with the help of narrowly focused research approaches, but rather focusing on a very central research question relevant to the respective field of application, but then investigating it with reliable results and not just with a kind of "accompanying evaluation."

Does this work with our different research culture that we have here in Germany compared to Anglo-Saxon countries?
Research culture is, of course, a crucial factor. That's why I'm talking about building a research infrastructure, not a data infrastructure, which requires a modern research culture to be able to conduct research not only quickly but also pragmatically when necessary.

What does fast mean?
Well, the British only needed nine days back then, from the initial idea of conducting such a study to recruiting and enrolling the first patient. But that can only be done with focus and if you can use an existing and proven research infrastructure.

A truly noble goal.
We still have a lot to do, don't we? I didn't take over as director of the institute to sit back and do nothing. I'm here to make the world a little bit better. Nothing more, nothing less.

With a sense of mission.
I also have a sense of mission: Above all, I want to use my life and working time meaningfully.

Dr. Kaiser, thank you for the interview. <

The interview was conducted by MVF editor-in-chief Peter Stegmaier.