Dr. Gilsdorf, during a presentation at the DKVF 2019, Ada founder Dr. Martin Hirsch stated that the Ada diagnostic app is not better than a good doctor, but better than a bad one. When will Ada be better than a good doctor?
That's not quite accurate. However, it's certain that Ada will inevitably continue to improve. The-
It is inherent to such AI-supported instruments that they generate new knowledge with every application and every input, which can be relatively well supplemented with existing knowledge. Furthermore, systems like ours—no matter how much new knowledge may become available—do not forget what has already been learned, whereas every human being—and thus every doctor—simply represses, forgets, or—when it is needed—can no longer recall old knowledge.

This is simply due to the limitations of the human brain.
Which is perfectly normal. Therefore, people, including doctors, should actively use such support systems, if they already exist. It's obvious that AI-supported systems like ours can be better, especially for rare or even very rare diseases that doctors never encounter, or perhaps only once, in their careers. However, this is an assumption that is currently poorly supported by evidence because – similar to a practicing physician – our system has also been confronted with such cases too infrequently.

Which naturally creates a bias.
This isn't just a bias, but a fundamental difference. An experienced doctor, with a good visual diagnosis and perhaps a good gut feeling, will quickly arrive at a result that at least points in the right direction. However, Ada has an advantage when it comes to this type of anamnesis using well-structured questions, because it not only incorporates the knowledge of thousands of experts, but also processes it in a very specific way. Ada's symptom analyses can therefore be a valuable tool even for good doctors, because doctors and artificial intelligence can complement each other very well.

Nevertheless, there is ultimately a fundamental difference between the – let's say eminence-based – anamnesis based on the experience and knowledge of a good physician and the evidence-based standard via Ada. How can one achieve a synthesis of these two worlds?
Ada was originally conceived as a purely diagnostic support tool to enhance medical diagnosis. For example, after registering, a patient would receive a (naturally disinfected) smart tablet from the practice manager, on which Ada is installed. They could then answer the questions in the waiting room, something virtually every patient can do intuitively and easily without lengthy instructions. Once finished, they would press "Submit," and the result would then serve as the basis for their medical history, not only for the treating physician but also—with their consent—secured in their electronic patient record.

You just said: originally. Has Ada moved away from that?
That's certainly a good approach, but Ada has evolved away from supporting medical diagnoses and, since 2016, has been an app aimed directly at patients. However, at the end of an assessment, patients also have the option of sending the results to their doctors as a PDF.

Instead of a doctor's letter, now a patient letter?
This is a highly structured medical history, captured by the patient via Ada, and supplemented with probabilistic statements. In addition to providing the user with the raw medical history information, the system also offers a description of their individual care journey – outlining the necessary next steps, where to turn for help, and how quickly they should take action. It would be beneficial to link this information to available appointments with various doctors. If this also saves the doctor time by providing them with the available medical history information, and perhaps even sparks important insights for diagnosis that they might not otherwise have considered, then we would truly be bringing these two areas together even more effectively – to the benefit of everyone: primarily the patient, but also the doctor.

According to Dr. Martin Hirsch's presentation at the DKVF conference, there were 15 million pre-diagnoses and 8 million users worldwide in 2019. What is the situation today?
We currently have over 10 million users worldwide with over 20 million assessments. In the US, we collaborate with a major healthcare provider, primarily in California. Here, in addition to symptom analysis, care navigation is supported, including—where appropriate—promoting low-threshold services such as telephone or video consultations. Of course, the recommendation can also be: see a doctor immediately or, ideally, go directly to the nearest hospital. Essentially, this introduces a kind of care triage, based on a probability level as well as the type and severity of the reported symptoms.

However, as is well known, symptoms are often not clear.
This is the major challenge facing every doctor, including ourselves. The symptom that prompted a user to see a doctor or use the Ada app doesn't, in itself, determine how urgently medical help is needed. For example, arm pain could indicate a strain or muscle soreness, but it could also point to a heart attack, which of course requires immediate medical attention. Therefore, it depends on the circumstances, which need to be clarified through further questions.
What public health analyses are possible with Ada?
Before joining Ada as Director of Epidemiology and Public Health, I was Head of Infectious Disease Surveillance at the Robert Koch Institute (RKI). Among other things, I was responsible for SurvStat, which for over 20 years has allowed users to individually access and create their own graphs of disease cases and pathogen detections that are subject to mandatory reporting under the Infection Protection Act. It's a fantastic, timely database that makes reported cases available to everyone after about three weeks – for a long time, it was unique in Europe. My role at Ada is twofold: firstly, to develop ways to improve individual care journeys using public health information; and secondly, to conduct population health analyses that we can aggregate from the pseudonymized data entered into Ada.

But what statements are possible now?
We have the significant advantage that, unlike all other available data sources – whether from doctors, pharmacists, or even health insurance companies – we can take a data-driven look at the time before a patient comes into contact with the healthcare system in any way. The traditional data system is inherently rather cumbersome: A person, or several people, feel unwell, go to the pharmacy, and get an over-the-counter medication. If that doesn't help, they consult a doctor, but by then they no longer feel unwell, but are already ill. The doctor then conducts diagnostic tests. If these are positive, the doctor will initiate initial treatment or, if necessary, transfer the patient to a hospital. Simultaneously, if it is an infectious disease, a report is submitted to the public health department, which then appears in the Robert Koch Institute's SurvStat database a few weeks later. And only much later does the case appear in the data records of the doctor and pharmacist, and much, much later still, often with delays of several months or a year, in the routine data of the health insurance funds that can be evaluated.

So, could Ada be a kind of early warning system?
Absolutely. Because the data available from all standard reporting systems only shows the tip of the iceberg – and unfortunately, only when it's far too late for a rapid prevention strategy. We, on the other hand, have both faster and deeper access to data on the evolving burden of disease. We can provide data much faster than others because we can analyze it in real time on the day it's entered.

Could Ada help to better control the Corona pandemic, at least in terms of symptom analysis over time? I sit there, coughing away, and the cake doesn't taste good anymore, but I wait 25 hours on the hotline of the nearest health department and then end up sitting in the waiting room of my family doctor, possibly infecting everyone else.
Digital health assessment tools like Ada could be centrally controlled and adapted in times like these, and then implemented relatively quickly across the board. This would help with symptom analysis, but also with timely communication of the best treatment pathways – for both patients and society.

Which would be useful not only in the case of Covid-19.
The German Infection Protection Act covers 60 diseases, but we have data on over 1,000 in our database, which can be evaluated both currently and retrospectively. For example, we could analyze when and where in the world the self-reported loss of smell and/or taste, now considered a fairly clear sign of COVID-19, was first frequently entered.

Why don't you do it then?
We are doing it. We just published a paper in the Journal of Medical Internet Research (JMIR). We were able to analyze how the Covid-19 measures in Germany, compared to the UK, affected people's health. To do this, we compared Ada entries in the month before the first measures were implemented with those from the first month of the complete lockdown and observed how users' behavior changed. The focus wasn't so much on whether they had more respiratory problems than before, or whether their sense of smell or taste was impaired, but rather on how the measures of a strict lockdown actually affected their health.

The first question that arises is: Is what your data indicates correct?
This is a question everyone who analyzes and publishes data should ask themselves. That's why we do it too. Therefore, our analysis wasn't so much about the results themselves, but rather about whether the insights generated from our data are comparable to the results of other, established data systems.

Were the results approximately congruent?
We are not entirely unhappy to say that our data did not reveal anything surprising, but rather exactly what we expected.

Which might sound a bit strange.
Not at all, because that simply means that the results of our data can replicate the results of other data sets almost perfectly, only much, much faster. For example, we observed significantly fewer gastrointestinal complaints during the lockdown, a phenomenon that has already been widely reported. We also saw that significantly less stress was reported in our population, particularly during the first month of the lockdown. We suspect this is because there was less work-related stress in the home office, where the work environment generally offers fewer work-related stressors than the office setting. Furthermore, app users reported better sleep, less hoarseness, and fewer ear problems during the period under review, possibly because they were no longer required to give presentations or travel by plane as frequently. However, more complaints such as dry skin were reported, possibly because people were washing and disinfecting their hands more often – which, of course, are all just assumptions and interpretations. To make more precise statements, further studies would need to be conducted based on the findings obtained.

However, all of these are at least observable collateral effects.
These are side effects of the measures ordered by the government to control Covid-19. Our aim is not at all to uncover or investigate anything that no one knew before. Rather, we are interested in exploring whether the quality of data we generate is truly usable for public health purposes. I believe we are well on our way to verifying this hypothesis. After all, this was our first attempt to use our data in an aggregated analysis to provide information on population health and evolving disease burdens.

Is there any further data that could confirm this hypothesis?
We are currently looking at data on influenza-like illness, or ILI for short.

A kind of fake flu or flu-like illness.
Exactly. The ILI is very well researched and monitored by many different surveillance systems, such as the RKI's web-based population tool called FluWeb. We have now compared the data from FluWeb for the last three years with our own data. The good news is: no surprises here either. We see almost exact matches with the waves recorded by the RKI, both over time across different years and across different age groups.

What do you do with it now?
We envision that the information generated by Ada could be used to support standard surveillance purposes in real time. This could be done for COVID-19, for the ILI, but also for many other disease entities that don't yet have their own surveillance systems. Such systems would be particularly interesting for countries that don't yet have their own surveillance systems. In large parts of the world, it would be advantageous for local health policy to have timely insights into how disease burdens are developing within the population. Ada could provide excellent data for such early warning systems.

It sounds exciting what the practical application of digitalization and AI might bring us. Consider Covid-19: Could it also be used to identify hotspots early on?
That's certainly something worth trying. Covid-19 has several typical symptoms, such as loss of smell and taste, which is quite common. If respiratory symptoms are also reported, the predictive value would likely be quite high, making a Covid-19 diagnosis rather probable. Of course, if someone only reports a cough, the diagnosis becomes less certain. Nevertheless, the app could then recommend the best course of action for the user.

Call your doctor, possibly get a sick note and stay home instead of going to the hospital or, if in doubt, continuing to go to work and possibly infecting many others.
Exactly, a care journey. It would certainly be possible to identify developing hotspots very early on.

Can you imagine it as similar to a navigation system that accumulates the movement data of thousands of drivers to predict traffic jams? If one car is driving slowly, the system doesn't report anything, not even if two, three, or four are doing the same. However, if ten cars are driving much slower than the speed limit or aren't moving at all, it's likely to be slow-moving traffic or a traffic jam.
The system is similar: individual data points are accumulated to make certain probability statements based on predefined limits.

However, one should also be aware that one can be completely wrong, as was shown by a case in February of last year when a Berliner pulled 99 smartphones through a street in a toy cart and Google then immediately displayed a fake traffic jam.
This example shows that the system itself works, even if it can, of course, be manipulated. Therefore, when setting up such systems, one must know what one is doing and, above all, what biases and limitations exist. Sticking with the Covid-19 example, this means that one could at least very quickly alert local health authorities to take a closer look at a specific region. However, this must not be done days or weeks later, but rather ad hoc.

In any case, that would be a tool that doesn't yet exist. Certainly not in real-time.
Similar approaches already exist, although not yet specifically for Covid-19. For example, the Robert Koch Institute (RKI) has created heatmaps of temperature and heart rate for Germany using its data donation app, which allows users of fitness trackers to voluntarily donate their movement data. Whether this is useful or not is debatable, but the example demonstrates the enormous potential of digitalization and AI. Of course, it is always crucial to carefully consider beforehand what information can be collected and how, in order to develop the most effective measures possible.

This statement seems to be central: the knowledge of how disease and probability models are modulated, on which algorithms can then be based.
Basically, we model diseases to obtain possible explanation probabilities for symptoms that a user enters.

For example, how did you proceed with the disease entity Covid-19, which didn't even exist until the beginning of 2020?
The advantage of Ada Health is that we have a large team of nearly 60 physicians who can extract and condense the available medical knowledge about a given disease entity from textbooks and published studies. This is easier with diseases that have been known and researched for a long time because there is a relatively large amount of evidence available. This was different with Covid-19, because the available knowledge about this disease had to develop from scratch. Furthermore, some of the knowledge is still in the pre-print stage, is growing almost daily, and is constantly changing as new findings emerge.

Since when can Covid-19 be detected in Ada?
We decided relatively early – as early as the end of January 2020 – to include Covid-19 in our model and integrate it as part of our larger model. The first queries were then possible in February. Of course, it was and remains extremely important to update this disease model very regularly.

How many assessments have been conducted on Covid-19 since then?
It's impossible to say for sure, because nobody enters Covid-19 as a symptom, but rather cough, fever, or loss of taste. Many people who suspect they have Covid-19 therefore run assessments without Covid-19 appearing as a possible cause for their symptoms in the results. However, our goal so far hasn't been to explicitly identify and diagnose Covid-19. Rather, our aim is to create a meaningful list of possible explanations so that we can then guide the user's behavior and next steps to maximize their health benefits.

To do this, you need to develop a consistent matrix whose modules can be filled according to specific indications.
The information matrix is a data structure that is the same for every disease, but varies in detail. Our matrix—as the entirety of all individual models—includes all the diseases we have already recorded and structured. Once this work is complete, numerous tests follow, using symptoms reported in case studies to determine whether the created model is suitable for capturing a disease and how it performs in relation to other, similar diseases. This is crucial because the predictions of the probability model depend on these results, as it represents the various probabilities with which a disease develops.
The frequency of symptoms is quantified. Of course, it's also possible that certain symptoms occur individually or in combination, while other symptoms don't occur at all. This, in turn, results in a ranking of possible probabilities for explaining the symptoms reported by users.

However, it will always depend on the quality of the expert input and how it is processed. In academia, someone who does this is called a data scientist. What do you call that?
Medical Knowledge Engineer.

Very nice: A kind of knowledge manager for medicine.
Not a manager, but an engineer. The English term "engineer" encompasses far more than just a medical-scientific writer or a knowledge manager. One writes a scientific text, which another then categorizes or assigns to a specific context. Our Medical Knowledge Engineers, however, are able to code the available knowledge about a disease and then transfer it into our model. This requires a team of many highly skilled physicians who then need intensive training to understand how the modeling process works. This expertise is one of our greatest strengths, for which we are highly valued by many of our partners.

Ada employs around 200 people in total. How do you finance all these people?
The motivation behind Ada is to disseminate medical knowledge to as many users worldwide as possible, thereby gradually improving health. Our approach therefore does not allow us to charge individuals for each assessment. For this reason, we are increasingly collaborating with healthcare system partners around the world who value symptom analysis combined with care triage and meaningful care navigation.

But not with German partners.
Our care navigation solutions are currently used mainly by foreign partners, for example from the USA or Tanzania.

With whom does your company collaborate in Germany?
In Germany, we are currently working with various partners from the life sciences sector, such as Pfizer or Takeda, to map further rare diseases in Ada.

Until mid-2019, you had a partnership with Techniker Krankenkasse (a German health insurance company). However, they terminated their cooperation with your company. At the time, it was alleged that Ada Health had transferred personal health data to both Facebook and the US service provider Amplitude. This was claimed, at least, by IT security expert Mike Kuketz and the computer magazine "c't".
Health data is the most sensitive data of all. At Ada, we are acutely aware of the special responsibility that comes with handling health data. Data protection is also a complex issue, and in our view, some aspects have been overlooked in the reporting. For example, providers of health apps use third-party services to ensure the functionality of their own applications or to guarantee the post-market surveillance required for medical devices. In accordance with the GDPR, we have binding contracts with these providers to ensure that our users' data is protected and secure. We continue to offer an optional Facebook login because many people in developing and emerging countries do not have an email address, and Facebook login is therefore the only way to use Ada – however, we do not transmit any personal health data to Facebook, as is sometimes claimed. The real problem was that our data protection and consent declarations were not explicitly and comprehensively designed for the collection of health data. Ada quickly rectified this, and the issues have been resolved.

What does the future hold for Ada in Germany?
We are definitely interested in increasing our efforts again.
to collaborate with German partners. We are confident that we will succeed in this, as the Digital Healthcare Act and the Hospital Future Act have set many things in motion. The opportunities opened up by these current laws – including in terms of reimbursement – will significantly advance companies like ours within Germany in the future. However, it must be clearly stated that other countries are still much further ahead in the digitalization of their healthcare systems and are therefore the more natural choice for larger partnerships.

Dr. Gilsdorf, thank you very much for the interview.

The interview was conducted by MVF editor-in-chief Peter Stegmaier.