>> Scientific modeling is booming in Corona times. Anyone who has spent their career using models to reveal astrophysical phenomena that remain hidden from direct observation should, one would think, be happy about it. But unfortunately we are not seeing modeling at its best during the Corona pandemic. Rarely in the history of science has there been a greater tension between the validity and political impact of predictions and the remarkable lack of openness and self-criticism when predictions do not come true. However, this insight is difficult because it would be too embarrassing if eminent scientific journals, science organizations, science journalists and, not least, political decision-makers admitted that the audience may have been sold “junk”. Of course, not everyone was blinded by naive model predictions, and many did not hold back in their criticism. It seems to me that it is time to initiate an even more aggressive approach to addressing the grievances. Firstly, the public has a right to ruthless honesty from us scientists, and secondly, we would do well to point out mistakes so that we don't collectively look like quacks in a few years and lose people's trust. This is not about downplaying the pandemic. Anyone who wants to protect people from illness and save lives needs an undistorted view of reality. It is no good if we make massive efforts to prevent five hundred thousand imagined deaths, but perhaps overlook the fact that with a more thoughtful approach we could actually have saved five thousand more. What could have been achieved if nursing homes had been able to use the money saved to more consistently set up separate wards for infected and non-infected residents and take other protective measures? What about other risk control strategies suggested by healthcare practitioners1? What possibilities have we perhaps overlooked because we have focused too rigidly on simplified models? But deciding that would go beyond the scope. The following will only be about how mathematical modeling fared and presented itself during the pandemic - as a small building block in the process of coming to terms with the last year and a half.
1. Kaleidoscopic inventory
From the outset, the prognostic and diagnostic modeling of the epidemic event was undisputedly one of the central pillars on which non-pharmaceutical interventions (NPIs) such as contact restrictions and business closures were based. We all remember Report 9 of the Imperial Group2 as a politically effective “bang.” The prediction was: 510,000 deaths in Britain without action. Despite the importance of predictive modeling and model-supported statistical analysis, there can be no talk of a consensus on the scientific findings, at least from a sociological point of view. An objection was raised against the Imperial Group's model that, when applied to Sweden by a team from Uppsala3, it overestimated the number of deaths by an order of magnitude. After his own analysis of case and death numbers, Nobel Prize winner Michael Levitt (Stanford) turned against the narrative of exponential growth and instead preferred fits with the so-called Gompertz function4.
The theoretical epidemiologist Sunetra Gupta from Oxford5 and the professor of evidence-based medicine, Carl Heneghan, who also works there, have also taken a position against the Imperial Group's line. As far as the effectiveness of NPIs is concerned, German groups from Göttingen and Braunschweig, for example, see a decisive effect of tough contact restrictions6; The analyzes by the group led by Ioannidis (Stanford)7 and the CODAG group at LMU give cause for skepticism regarding the effect size8. The emeritus Aachen epidemiologist Ulrich Keil9 is also critical of the reliability of the models. The group of authors led by Matthias Schrappe recently attested that modelers were naive in their approach to a complex system such as the pandemic10. An exhaustive presentation of opinion among scientists beyond this kaleidoscopic impression could fill volumes.
From the beginning, the dissent also manifested itself outside the classic channels of intra-scientific communication. The scientific debate has spilled over into newspaper interviews and talk shows, on YouTube and on Twitter, and often enough journalistic “fact checkers” have become arbitrators. Michael Levitt was attacked in the “FAZ” for operating “beyond the core expertise”11, the Göttingen MPG group came up with an unflattering epithet in an article in the “Welt”12. Under the title “The Great Modeling Controversy”13, the “Berliner Zeitung” highlighted the conflict between Matthias Kreck, winner of the prestigious Cantor Medal of the German Association of Mathematicians, and the groups from Göttingen, Braunschweig and Berlin, which, interestingly, occurred despite an affinity in the original ones objectives have been developed. In this article, another modeler, Dr. Mohring said that, in his experience, there was no in-depth exchange about the problems and strengths of individual model approaches in Germany. We don't even want to talk about how the debate on social media, led by scientists from different factions and their supporters, is getting out of hand.
2. Wrong balance?
Even those who understand nothing about the supposedly deep equations in the models will rightly sit up and consider whether they are dealing with a “contaminated” scientific field. It may be that we are dealing with an exaggerated controversy and a “false balance” between the orthodoxy and the heretics who are driven either by ignorance or sinister interests - an interpretation that appeals to government-affiliated modelers and their allies such as Germany's star virologist Christian Drosten.14 Maybe civilized scientific discourse has also collapsed because all sides cannot see the forest for the trees. Be that as it may, there is reason to investigate the matter and to critically examine the political advice provided by epidemic modelers.
Because the claim of a “false balance” is not convincing if you take a little criminal experience into account. There is sometimes a “false balance” in scientific debates. It didn't surprise anyone that Richard Lindzen or Freeman Dyson were playing devil's advocate on the issue of global warming, but unease about the reliability of epidemic models can also be found among much more unconventional and grounded characters. And the overconfidence of researchers from outside the field cannot be the problem either. When two groups in Germany advance from blank slate to government advisors within a few months in the field of epidemiological modeling (unthinkable in climate research), while a Nobel Prize winner like the biophysicist Levitt and Kreck, the recipient of the Cantor Medal, turn out to be too stupid to make a lateral move , while if a master of medical statistics like John Ioannidis and a holder of a prestigious Royal Society Wolfson Fellowship like Sunetra Gupta were to suddenly “derail,” then that would be sensational and worthy of investigation in itself. Even an outsider like Tomas Pueyo, without any academic merit in the subject, was received by Drosten in the NDR podcast as a serious modeler and analyst15. And how can it be that the non-expert Kreck is dismissed by the RKI modeler Brockmann and quoted by an eminence in mathematical epidemiology such as Odo Diekmann and explicitly mentioned with thanks for the fruitful exchange of ideas16?

One cannot hope that such a complicated and politically charged situation can be resolved on normal channels - through regular research work, conferences and so on - in the foreseeable future. Only over long periods of time does the truth prevail in the regular scientific world over the fashions of the moment, the inertia of those who are not involved and the wagon-castle mentality of those who are afraid of scratching their self-image. In any case, people care less about the trials and tribulations of the past than about the new, apart from the history of science, which eke out a rather meager existence far away from the large pots of third-party funding. In such an important matter, the public deserves better; She deserves to know what was resilient about the models in whose name the executive branch directed her for months. This needs a different format, just as Stephen Jay Gould's fight against the incorrect measurement of people17 could not only be aimed at specialist colleagues. In a sense, it takes a forensic approach.
As someone who knows a little about the mathematical modeling of complex systems, I want to contribute to this - not necessarily as a completely dispassionate analysis, but rather in a subjective way, as a scientist who wonders. We've suppressed the frown long enough.

If you first came into contact with the idea of exponential growth of epidemics at the beginning of 2020, this concept might seem obvious. For example, with a basic reproduction number of three, an infected person infects three other people within a certain time interval, then nine in the next generation, then 27, and so on. The whole thing repeats itself until you get close to “herd immunity” and the possible follow-up contacts have already been used up. Using simple equations at the level of intermediate mathematics, this herd immunity threshold can be calculated from the reproduction number. Of course, it was clear to anyone with mathematical training at the time that they had to expect much more complex behavior in real epidemics, otherwise the field of mathematical epidemiology could have long since been abandoned because of the complete solution to the problem. If you delved a little deeper, you found so-called SIR models in which the population is divided into susceptible, infected and recovered people, for whose shares differential equations are then set up, although some embellishments of this approach are possible. Much of what was thrown onto the market with publicity was limited to this type of model, for example the study by Dehning et al.18, which sought to confirm the decisive effect of the lockdown in spring 2020. This approach, which does not change much of the naive image of exponential growth, is of course almost a hundred years old19 - I remember well from my high school days a first encounter in a typewritten textbook about differential equations.
The fact that realistic epidemic models could be built so cheaply seemed suspect from the start, but given the information available, people were happy to give the thesis of impending exponential growth a leap of faith. But soon there was increasing evidence that reality was much more complex. Even as the first wave was subsiding, the dispute arose as to when and why the famous R value had fallen below one. On the international scene, Levitt came onto the scene with the claim that the SARS-CoV-2 outbreaks were not growing exponentially, but rather according to a so-called Gompertz function.20 The breaking of the exponential growth was largely attributed to non-pharmaceutical interventions. This is understandable, because a reduction in contacts must undisputedly influence the infection process - but it is still a logical short circuit. In order to attribute the slowing of the infection rate solely to the measures, one would have to - to paraphrase Sherlock Holmes - exclude all other possible explanations. Most of the time this was done sloppily - for example by referring to the “anticipatory obedience” of citizens who had already reduced their contacts before the lockdown21.
Scientifically, this approach is simply lazy. Instead of being satisfied with an ad hoc explanation, no matter how plausible it might be, after this first surprising development there were actually two indispensable tasks to be completed: firstly, the assumptions behind various models used had to be ruthlessly dissected, and secondly, in parallel to search the specialist literature for similar phenomena. This should have been easy for the high-profile epidemic modelers, but they were apparently neither encouraged to do so by politicians, nor did they do it on their own initiative, nor were inquiries from colleagues fruitful. Why couldn't anyone answer Levitt as to what was wrong with his numbers22 and why they couldn't use his method?
The answer could have been, for example: "Yes, we know of epidemics that do not grow exponentially, but that cannot work with SARS-CoV-2 because..."
Nothing of the sort has happened after initial doubts about the exponential growth thesis on the part of government-affiliated modelers.

This is surprising, because with a little experience with complex systems you can piece together in a few hours why the “official” narrative might be too simple, and a few more hours of literature research are enough to confirm your suspicions. A quick analysis shows: Obviously, one cannot assume that every infected person infects the same number of subsequent contacts and that the subsequent contacts are randomly distributed throughout the entire population. The number of follow-up contacts will vary; apart from the family, they may often be in the same age cohort. They will not spread randomly across the country, but will move more often in a smaller area and often form local clusters, whereby proximity can be understood in a geographical sense or in the sense of an abstract (and also constantly changing) network of contacts. The business traveler will likely have more potential follow-up contacts than the widow who lives alone. Beyond the heterogeneity of such sociological factors, there will also be complications on the medical side. Some people don't become infected at all for some unknown reason, others excrete the virus for a long time and intensively and become a super spreader, and you also have to take into account variable environmental factors (UV radiation, temperature, humidity, etc.). Anyone who is familiar with complex systems and only recognizes some of the possible complications immediately comes up with various ideas to better represent reality: You can divide the population into more subgroups and replace the reproduction number with a matrix to describe how often someone from age group X infects someone from age group Y, or divide people even more finely according to the number of contacts they have. The “local” nature of the spread can be modeled as a so-called reaction-diffusion system, or it can be represented using more or less sophisticated models of contact networks.
You can bring in all sorts of arcane mathematical concepts that are incomprehensible to laypeople - renormalization group theory, the theory of fractals, bifurcation theory and the like. Of course, what matters is what comes out at the end: In more complex models, epidemics do not have to grow exponentially until almost the end - even without contact restrictions. How, for example, clusters in contact networks can slow down exponential spread is shown very simply in Fig. 123. And the size of an outbreak cannot be calculated simply from the base reproduction number.
In May 2020, I myself spent a free hour just for fun trying to see whether the first wave in Germany could be reproduced with a few assumptions about the contact patterns of the population, without assuming any effect of the “lockdown”. It's an easy exercise (Fig. 2), from which you can only draw one conclusion: with a little creativity, you can probably - to paraphrase Radio Yerevan - provide almost any explanation for any wave of the pandemic.
Of course, that doesn't mean that you can bend reality to whatever you want. Only when a model survives repeated confrontation with observational data does it gain credibility. However, this alone is not enough, especially when models have a significant number of parameters as adjusting screws to bring them into line with observational data. Extracting model parameters from the observation data itself is unavoidable in epidemic models, but a sense of “parsimony” is required so that predictions do not turn into completely arbitrary interpretations of what is happening. With too many parameters, perhaps not even clearly determinable, the explanatory value of a model may approach zero. But even if a mathematical model, for example, predicts the daily national case numbers well with a few, very clearly determinable parameters, you still haven't done your homework.
Once the first test has been passed, you should look a level deeper into the data and check whether the model predictions or assumptions are still confirmed or whether the complications discussed previously play a role. For example, can I actually ignore heterogeneity effects and assume a completely mixed population?
You can gain clues if you break down the infection process more precisely, for example by district. If you do this for the number of cases in the spring wave of 2020 or the number of deaths in the winter wave of 2020/21 (Fig. 3), then you will notice a significant temporal asynchrony in the individual districts, and you have to think very carefully about whether to use averaging over the entire country can simplify without coming to wrong conclusions.
The county data also show significant variation in population mortality (Fig. 4), which needs to be explained. A possible explanation for the significant heterogeneity in the spatially resolved data is that it was caused by different decisions by individual district administrators about the fine-tuning of measures at different times. But one must also take into account that the natural infection dynamics inherently produce such fluctuations, which must be taken into account in models and when interpreting the data24. The fact that a system shows structures on different scales (from the family to the village to the district, etc.) and fluctuations do not average out as expected immediately awakens associations with similar behavior in other phenomena such as forest fires or turbulence.
Since activity maps25 and the distribution of case numbers26 for influenza reveal a similar picture of significant fluctuations on meso- and macroscales, despite all the differences, the suspicion arises that natural heterogeneity effects can also be expected with SARS-CoV-2. This glance at heterogeneity effects, which only scratches the surface, gives a taste of how one has to painstakingly examine data in order to get closer to reality. It becomes even more strenuous when you first have to collect the data. Example: If my model says that the R value has fallen because contacts outside the family in leisure time were prohibited, then I should check the actual infection routes before and after the contact restrictions - even the most gifted theorist cannot tell something like that from his desk carry out astute calculations and literature research.
Those familiar with the subject know well that solid data collection is a crucial and unfortunately neglected challenge27. But only such a “look with a magnifying glass” can provide truly robust insights. If you instead throw readily available data into an automated statistical machine and leave the knowledge production to the computer, you have to be prepared for mistakes. “Looking with a magnifying glass” will usually reveal details that require more complex models to explain.
A dilemma can easily arise. Firstly, you may come up against the technical limits of available computers, secondly, the necessary parameters of more delicate models may not be practical or can only be determined with difficulty, and if they do, then thirdly, parameter uncertainties may increase to such an extent that it is no longer possible to achieve a higher level of predictive power. Ultimately, overly complex models become so opaque that it may no longer be possible to see where the pitfalls lurk. Sometimes you'll do better with a simple model for short-term forecasts, as long as you don't force it down the wrong path with problematic assumptions (like continued exponential growth) and keep the limited forecast horizon in mind - but this requires extreme caution.
Because of all this, one is quickly faced with the choice between the unattainable goal of maximum realism on the one hand and unsatisfactory simplifications on the other. In this context, one must point out the pitfalls of so-called “agent-based models”, which are the hobbyhorse of the Imperial Group, among others28. In such models, an epidemic is simulated using individuals who drive to work every day, meet friends, travel, etc. and become infected according to certain rules.
This approach may come closest to the popular ideal of a simulation, but offers numerous opportunities to overlook difficult effects. For example, it is relatively easy to determine how the distance to the workplace is statistically distributed, or how often age group A has contact with age group B per day29. But it is much more difficult to measure whether these contacts pose a risk of infection (walking together or drinking beer indoors).
In this context, other disciplines speak of proxy data: you actually measure a slightly different quantity than the one you actually need in a model and hope that there is a sufficiently close connection. This works as long as this connection is well calibrated and understood, but it is always potentially a source of error. There are also all sorts of niches where implicit simplifications can be overlooked. An agent-based model can fairly easily represent that X and Y both travel to work every day at company Z and can potentially infect each other there, but not that .
Whether this makes any difference in the result remains to be seen. Perhaps Company Z with 1,000 employees behaves more like 20 companies with 50 employees, resulting in less effective mixing and slower spread of infection than expected.
However, the problems mentioned are not limited to complex models. For example, one can account for social distancing in a simple epidemic model by coupling the probability of transmission to mobility indicators from telephone data as a proxy. This will capture the effect qualitatively, but one must be careful with quantitative predictions. A truck driver who just stops quickly at the gas station, eats his lunch alone in the parking lot and always stays outside when loading and unloading will drive the mobility indicator up slightly, but will not contribute to the infection rate. On the other hand, the mobility indicator says nothing about infections between residents of a nursing home. It should not be assumed lightly that all of these subtleties will be averaged out when you take a rough look at the entire population or even just a district. And even if they average out approximately: If you repeatedly neglect 50% effects based on acceptable approximations, you will quickly end up being off by an order of magnitude.
If there is then a strongly non-linear dependency on the parameters, the error margin increases accordingly or the forecast horizon shrinks. Take five percent uncertainty in the probability of transmission, five percent in the contact rate, five percent in the duration of the infectious phase, and an overly simple forecast about the winter by extrapolating the current conditions easily turns into a blind cow game. Either the areas of uncertainty explode, or you end up in a regime where other factors suddenly play a role and model assumptions collapse.

If mathematical epidemiology had overlooked for decades what a theoretical physicist can come up with in a few hours, I would have been suspicious and doubted myself back then. A look at the literature immediately shows that all conceivable complications and their effects on the dynamics of epidemics are well known, and the fact that they can be understood very quickly with a little training naturally has to do with the subcutaneous absorption of concepts that are about have worked across scientific disciplines for decades. Only a few highlights are mentioned here, and the reader may forgive the sprinkling of technical language: Decades ago it was recognized that the famous R0 in complex SIR models does not describe the entire transmission dynamics, but, to put it simply, only the greatest intrinsic value of a “reproduction matrix “30,31,32 – although I cannot unravel the research chronology here in a meticulous way. Models are repeatedly discussed in which the number of new infections decreases faster with the size of the outbreak than in classic SIR models in order to take into account slowing effects caused by clusters33,34,35.
Nor can one generally determine the size of the outbreak (“herd immunity” in the naive sense) using the classic formula
Estimate 1-1/R036. So-called metapopulation models also show non-exponential growth and sometimes smaller outbreaks than one would naively expect37,38. Network models also often result in a braking effect caused by clusters.
And as early as the 1980s, the Austrian physicist Peter Grassberger from the Jülich Research Center was working on the problem of epidemic spread on a two-dimensional grid in which the infection spreads in the form of a front and the number of cases grows over time as a power function instead of as an exponential function39.
Here's an insert: Ironically, Grassberger's picture of an epidemic as a so-called percolation process was taken up by Christian Drosten in his NDR podcast, and then, after a largely correct introduction, was misunderstood in an almost ingenious way and turned upside down. The fact that Drosten, ignorant of Grassberger's work, claimed that the principle of percolation from infection ecology “has not yet been transferred to much, especially in the German-speaking world” and, when asked, revealed no knowledge of the formal-mathematical basics is bitterly humorous.
But the theory does not only provide cause for optimism. Unlike simplistic models, outbreaks cannot be completely suppressed in network models with certain properties40,41. Subtle properties of the network structure can have consequences for the optimal immunization strategy42,43. Apart from the medical-pharmaceutical problem of the limited and decreasing effectiveness of the available vaccines against the further spread of the virus, one must also make some critical footnotes about the promise of “herd immunity” through vaccination from the perspective of mathematical epidemiology. The interaction of declining vaccination protection and subsequent reimmunization through mild infections itself also offers room for surprises due to potentially chaotic dynamics that limit the horizon of forecasts44. And you won't just find gray theory in the literature: ideas about non-exponential growth are usually motivated by observations.
A particularly interesting example: When fears about the exponential growth of AIDS became widespread at the end of the 1980s, the number of cases actually grew cubically. One might naively explain this by the educational and prevention strategies adopted, but Stirling Colgate, a pioneer of complex systems theory, pointed out that such non-exponential growth could also have something to do with the distribution of sexual contact frequencies45. History doesn't repeat itself, but it rhymes. Why were all the complications mentioned with SARS-CoV-2 suddenly meaningless?
Why didn't the government's experts protest when Markus Söder lectured in the Bavarian state parliament about exponential growth at best at an intermediate level46 and Christian Drosten used the example of compound interest in a tweet47 in order to once again illustrate the growth of the epidemic in a dispute with critics? Why did no one see the connection between Michael Levitt's non-exponential Gompertz model and previous work in the literature, but simply brushed it off? Didn't the modelers know better, didn't want to say it - or is SARS-CoV-2 actually so magically different that everything becomes simple again?

There are some arguments against such a miracle and for a mixture of operational blindness, ignorance and selective communication with the public. Many of the prominent players undoubtedly know very well about the ramifications of mathematical epidemiology, even if they sometimes lack an overview of the bigger picture. Some people will simply have had the experience that conceptually
Deep works based on methods from quantum field theory do not win a big pot of popularity among the general public. Some people may even have specialized in narrow aspects of contact networks, but will sometimes overlook the fact that the detailed structure of contact networks also has a non-trivial impact on very simple questions such as the size of the outbreak and the development of production numbers over time.
One should not underestimate the talent of scientists for pigeonholing: someone may know how to “untangle” the network of international transport connections48 in order to be able to understand the dynamics of an epidemic as the spread of an abstract wave front, but perhaps at the same time be aware of network effects on small scales within of a country do not give enough thought. And some newcomers may seriously believe that the subtleties of the infection process will magically be averaged out in the end and that you automatically end up with the simplistic models if you only want to take a coarse-grained look at the epidemic dynamics, e.g. B. on the national number of cases. Even in the last year and a half, however, there are nuanced voices in the literature. Gabriela Gomes, professor of mathematical epidemiology at Strathclyde, pointed out that individual variations in susceptibility can lower the “herd immunity threshold”49, as did a team from Stockholm and Nottingham in the journal Science50.
A group at the MPI for Complex Systems found a similar behavior due to certain heterogeneity effects51, and computer scientists from Saarbrücken pointed out the influence of the detailed contact network structure on the infection process52. These quiet counterpoints are not unexpected given what has been said above, but they are largely ignored and, at worst, hostile in public discourse.
It is not apparent to me that they have been shown to have fundamental misunderstandings of the specialist literature. You are not alone: from many of the quieter modelers and statisticians in Germany you can hear a profound sense of the complexity of the problem, and a discomfort with an overly forceful approach - if you listen to them. Ironically, even the RKI used an arbitrary adjustment screw in early model scenarios to take into account that cluster effects could reduce the maximum number of infections by a certain factor53.
People once suspected that their forecasts could be significantly too high. Why known uncertainty factors should then have disappeared was never communicated. This also raises the suspicion that people – consciously or unconsciously – do not want to see problems with the models. Or even worse: you see them, perhaps even discuss them secretly in professional circles, but sell your own product to the public without the necessary warnings - a Janus-faced approach that would be nothing other than the most outrageous form of lying.
The state-affirming modelers hardly work at a much higher level of competence than their critics so that they can reject their objections with an ipse diximus. In the first work of the Göttingen MPI group54, which found three levels in the R value in the first wave due to government measures and attributed the decisive role to contact restrictions in turning the first wave around, there are, for example, inconsistencies that are not noticeable to the layperson, but are worry the specialist. It is surprising, for example, that in a follow-up analysis by the authors, with a better data set (cases reported according to the time of the first symptoms) and the “most realistic” model, the decisive effect disappears. If a supposed effect becomes smaller upon closer inspection, alarm bells should ring for every scientist.
The attempt to statistically extract levels in the R value and other parameters such as reporting delays from the number of cases is also technically suspect in other respects. For some of the parameters, the assumed prior probability distribution and the posterior probability distribution obtained from the data using so-called Bayesian inference barely overlap.
Translated for the layman, this means, for example: A reporting delay of around 8 days is assumed, but because the data probably wants around 14-15 days, the statistical inference machine agrees on 11.4 days. In this way, conclusions about the temporal sequence of events (first lockdown and then R value below 1 or vice versa) become dependent on subjective assumptions - if you believe strongly enough in a short reporting delay, then the R value will fall later. Anyone who wants to answer on such a basis whether the R value fell below one before or after the contact restrictions needs a lot of courage.
And just trying to figure out structures on a time scale of less than a week from the very noisy reporting numbers is a serious stomachache if you have a little experience with time series analysis. I have already mentioned before that the complete neglect of heterogeneity effects in such an approach harbors risks: If the number of cases is still rising in half of the districts and is already falling in the other half, how do you then decide whether a certain nationwide measure should be taken? exactly one date made the difference?
If the government-affiliated modeling is correct and robust, despite all the gut feelings, its lawyers do little to defend it in direct discussions with critics. My own experiences may not be representative, but they are worth mentioning. For example, after I expressed methodological concerns about the approach of Dehning et al. in an eLetter for the journal “Science”. made, the group replicated it civilly in the same forum, linked additional analyses, and contacted me politely via email, thanking me for my comments and inviting me to provide further feedback. Just as friendly and professional, I explained my assessment in more detail and also gave constructive advice on what to do to make it better - only to never hear anything again afterwards.
I had a friendly exchange with an unnamed employee at another institute in response to an inquiry about uncertainties in epidemic models. The information was very polite, although at first there was a somewhat appeasing and improvised, but by no means dishonest, “whistling in the forest”. In a second round of emails, my contact confirmed the “impression that the desire to use a particular model and the desire to help often plays a stronger role than the serious examination of model assumptions and the validity of the models used.”
This assessment by an “insider” is probably not alone. In the literature, attempts to criticize maneuvers in the wake of epidemics have not been made frequently. But some examples can be found. Even in previous epidemics, after sufficient probing in the context of anonymized interviews, modelers expressed themselves as being significantly more cautious than was reflected in their external representation to political decision-makers55. An obstacle to reliable predictions repeatedly emerged. Validation, i.e. a real test of the models on real data, never actually takes place. Although one can find many cases in the literature where models are subsequently adapted to observational data, making models “fit” is by no means a validation that provides confidence for the next predictions.
The situation may be better for small-scale problems, such as infections in a boarding school56, but in a pandemic of the current type there are considerable doubts as to whether model uncertainties are correctly assessed and communicated. Even more suspicious is the public handling of predictions by government-affiliated modelers that didn't come true, and there were quite a few. Applied to Sweden by another group, Neil Ferguson's simulation code overestimated actual deaths by an order of magnitude. In Germany, the MODUS-COVID report from March 19, 2021 put 157 incidences of more than 2,000 in May in the room. Even in more optimistic alternative scenarios, the incidence in May was around 500, several times higher than reality. Simply explaining away this discrepancy with behavioral adjustments in the population and similar arguments58 is just embarrassing.
When a model misses reality like this and the first thing you ask yourself is whether you have made a fundamental error in your calculations, you have to put up with the question of scientific competence. Because, contrary to all apologetic representations, scientific modeling is all about hitting the target when you put such concrete numbers on the table. If you wanted e.g. For example, if you only explain qualitatively that you fear a massive increase and that three rapid tests per week at school can avoid this, then you should simply say so. One sentence would be enough and one would have done one's duty: "The model qualitatively shows the effect described, but the absolute numbers can be subject to an uncertainty factor of five to ten." If instead, in the event of failure, one relies on circular59 arguments like this that serve to immunize hypotheses If the “prevention paradox” is resorted to, it smacks of a protective claim.
It may sometimes be true that a predicted catastrophe does not occur because politicians and the population reacted to the prediction in a timely manner. But this explanation is not logically compelling. If it is true, then you will usually be able to find a huge amount of convincing evidence for it. At least you should then recalculate your model with new data in order to make a prevention paradox at least plausible, without silently turning screws to cover up mistakes. Extracting yourself from the affair with oracular vagueness – that stinks.
If the assessment of pandemic modeling presented is correct, it should not be seen as a fundamental criticism of mathematical epidemiology. In the context of complex systems, the value of models is often measured more by their contribution to conceptual understanding than by predictive predictions. For this purpose, highly idealized models are useful and even necessary. There is no doubt that extremely intellectually demanding and valuable research is carried out in mathematical epidemiology. Pioneers such as Grassberger, Hethcote, Diekmann, Newman and many others have shone with outstanding, technically demanding work. In the literature one finds exquisite works of great mathematical depth that are entirely committed to theory. But the need for abstraction also creates a gap with reality that a good modeler must address with an extraordinary degree of judgment, healthy skepticism, and an open look at the real data in order to avoid setting himself up for a trap. Many experienced and cautious representatives of the discipline and the classical works are well aware of this.

As an individual scientist, in such a muddled and rapidly evolving situation, one cannot provide a complete and completely impartial analysis of the pandemic modeling over the last year and a half. Under normal circumstances, it can easily take six months to a year, in addition to day-to-day duties, to produce a solid review article on a reasonably narrow topic in the field of scientific modeling. Such a leisurely review of current events in a quiet tone of cool professionalism would come too late and would not reach the right audience. In order to get the necessary discussion going, some of us have to play devil's advocate and we also have to openly express the suspicion that there is a lot wrong with the pandemic modeling. In view of the massive interventions in people's lives, which were largely based on model predictions, this discipline cannot be sacrosanct. The advance of trust has simply been used up. Of course, the points of conflict cannot be resolved with essayistic polemics and manifestos. What is needed is a commission of inquiry to scientifically model the corona pandemic, and not in the sense of a task force to develop the ultimate pandemic simulation.
The look must finally turn to the past, towards criticism of maneuvers: What can the models really do and why have they sometimes missed the mark? What has been confirmed and what has turned out to be a misjudgment? What mistakes were made? Putting the models on trial, so to speak, is not an imposition, but rather more than appropriate at a time when the self-purification processes of science are in doubt and certain prominent actors are speaking out about a culture of censorship60. This does not mean starting with blame, but rather examining the matter scientifically in retrospect and determining, in the spirit of Ranke, “what it actually was like” in order to learn from it together.
Science journalism has also shown itself to be too uncritical, despite a few laudable exceptions. The way the work was done was sometimes shocking. When I emailed an award-winning science journalist last year about his overly uncritical approach, he was initially willing to talk, “but better on the phone,” and not without emphasizing, in a manner of superiority, that he “knows a little about virology , epidemiology and immunology”.
He also assured me of his willingness to exchange ideas after I told him, among other things, sent a 55-page review article61 in “Reviews of Modern Physics” as a basis for discussion; He will get in touch in two days, after which he will have read the paper. After that: radio silence. To admit that he had overreached himself because he lacked the necessary mathematical foundations would probably have damaged his self-image too much. Such a reaction still shows class, because the simple forest-and-meadow fact-checker simply doesn't react at all when a scientist gives him a heads-up.
Would-be people who check the facts for their ideological fit and not for their truthfulness, and who probably use their censorship work to process personal traumas from their early semesters, are of no use for fact-finding purposes. The
Historical experience shows that a commission of inquiry is the appropriate format for dealing with a major disaster or controversy. It takes investigative questioning of witnesses and experts and active collection of evidence to bring even the uncomfortable to light, to break through the web of loyalties and friendly consideration, and to learn the right lessons. This is how it was handled when the Tay Bridge collapsed, as well as after the sinking of the Titanic and the explosion of the Space Shuttle Challenger.
It would be a scandal that the very models in whose name citizens were reduced to recipients of orders for a year and a half were allowed to evade this. Of course, mathematical epidemiology itself must also have a voice in such an undertaking. But the pandemic modelers do have to listen to critical questions about their work, even if they dislike it as a shaking of their throne. The increased desire to educate and the increased objectivity associated with the external perspective have proven to be valuable time and again throughout history. Among the members of the Rogers Commission to investigate the Challenger explosion were not only space engineers, but also the theoretical physicist Feynman, a solar physicist from Berkeley, an active Air Force general and Chuck Yeager, who was the first test pilot in the US Air Force in 1947 had broken the sound barrier and never attended college, as well as two lawyers and career diplomats, Rogers and Acheson.
During the American investigation into the sinking of the Titanic, the legendary Senator Smith did not shine with his specialist knowledge, but perhaps that was precisely why he maintained the course of ruthless investigation. In the words of GK Chesterton: “It doesn’t matter whether Senator Smith knows the facts; What matters is that he really tries to find out.” And if modelers find time to write popular science books during the pandemic62, they can very well be expected to answer questions from a competent investigative committee. Not shirking responsibility is also in the own interest of modelers and science as a whole.
If, in the best case scenario, it turns out that everything humanly possible was done and no avoidable errors were made, this will strengthen public trust in science in the long term.

In the end, my suspicion that pandemic modeling is largely flying blind may turn out to be unfounded. But even then, their prominent representatives have to blame themselves for getting caught in the crossfire of criticism. Because if they really worked professionally according to all the rules of the art, brought to the table not only profound knowledge but also the appropriate judgment, communicated with brutal honesty and always looked beyond their own horizons, then they concealed this excellently.
Where was the comprehensive assessment of the state of the discipline, where was the digest that could have enlightened decision-makers who were less mathematically versed about the depths of modeling? Where were the lectures that did not market one's own work, but rather took the audience away from the basics to the state of research?
At least the pro-government modelers in Germany didn't give the message that they had looked around before rushing forward. One did not get the impression that, given the crisis situation, they were trying to be more than scientists and lobbyists on their own behalf. They could have demonstrated judgment, wisdom and leadership in many ways. Instead of explaining in a differentiated manner where the models only illustrate qualitative connections, where they may be quantitatively reliable, where their uncertainties and limits lie, people reacted with intransigence according to the maxim: The simulations are to be believed, and if they are not correct, then that is it not our fault.
One could have recognized that, by their nature, phenomenological models cannot simply trump practical medical experience, as reflected, for example, in the thesis papers of the Schrappe group63. Often (though not always) in mathematical epidemiology, theory only translates into abstract language what has long been familiar to practitioners64. The modelers could have shown that they were aware of the problems in recording and controlling complex systems. Were they not disturbed by Hayek's reflections on the limits of economic models, Clausewitz's thoughts on friction in war and Moltke's dictum that no plan survives the first contact with the enemy? How could one seriously consider, on the basis of highly simplified simulations, achieving the goal with a few weeks of tough measures in the spirit of No Covid? A “child’s game”65?
One thought should have flashed through the mind of anyone with a fair bit of historical education: Schlieffen Plan. As much as the dream of the predictability and plannability of the world has advanced science and technology, as triumphantly as it has sometimes been realized, more humility in the face of reality and history would do us good at this point. <<

Correction (12/20/21): The epidemiologist Ulrich Keil comes from Münster, not Aachen