The connection between digitalization and health services research? Part 1
On the occasion of the 2019 SpringSchool of the German Network for Health Services Research in April 2019, Christian Klose (Department "Digitization and Innovation" of the BMG) presented the BMG's roadmap for the development of the Digitization Act (1). This presentation is valuable for several reasons, although the reason for the connection between "digitization and innovation" is not immediately obvious. The value becomes apparent in the description of the determinants to be taken into account*. The Institute of Clinical Economics (ICE) eV is therefore establishing a new working group that deals with the practical application of technical advances in telecommunications in health services research. In this first part of our report, the preliminary work of politics is briefly mentioned and the preliminary work of science is described: The differences between clinical research (KF) and health services research (SR).
>> The differences between KF and VF lie in the different goals of the two research directions, the different study conditions under which these goals can be achieved and the different instruments that are required to achieve different goals under different conditions.
Advance payments from politics: linking digitization and health care
Research-compatible patient records
Under the heading “Research-compatible patient records,” terms such as “electronic patient record (EHR),” “data,” and “interface” appear without any explanation as to what specific data is involved. The resulting problem can be illustrated with the term “big data”. If one assumes that “big data” describes an infinitely large amount of data, it becomes plausible that it makes no sense to offer this data without selection. The added value that can be generated from “big data” only arises through a targeted selection of specific data that is linked to other data on the basis of a logical consideration or a concrete hypothesis. Without a specific selection or precisely formulated hypothesis, “Big Data” initially remains just an infinitely extensive offering that cannot generate the expected added value per se. Experience will show what information can be extracted from “big data” or collected afterwards, because only the specific question or hypothesis reveals the need for this data.
So far, only the German Diabetes Society (DDG) has attempted to answer the question at the beginning of 2019 as to which values should really be considered and documented in acute treatment cases. To this end, the DDG is introducing the apparently new term “standard” for data and defining a data structure concept in a so-called “DDG diabetes file” (2).
However, even this "standard" may not be sufficient to be able to answer all questions that can be asked about a defined clinical picture, because the data required for an answer can only be defined by the precise questioning. This is also the main difference between a clinical registry and an experimental clinical study: the clinical registry first collects the data and then asks the question; the experimental clinical study first asks the question and then collects the data (3).
Human-to-human treatment remains the gold standard.
Whether it concerns the general conditions of a video consultation or the application of artificial intelligence (AI) is ultimately irrelevant. Ultimately it's about the content that needs to be communicated. An equally important core aspect of digitalization is communication, i.e. a relatively complex process that contains significantly more aspects than just the transmission of content. Although the information highway is needed to transport information from “A” to “B”, what is crucial is whether this type of communication is effective, i.e. whether it can ultimately provide the expected benefits. When we talk about communication, we should stick to naming Watzlawick's five axioms (4) in order not to go beyond the scope of this article. Nevertheless, it is recommended to read the meaning of these axioms:
1. You can't not communicate
2. Every communication has a content and a relationship aspect
3. Communication is always cause and effect
4. Human communication uses analog and digital modalities (note Watzlawick's definition of "digital")
5. Communication is symmetrical or complementary
The aspects of communication must not be ignored because, according to Klose, digital transformation is less a question of political regulations and more a question of “wanting and designing” and “wanting” is significantly influenced by communication. Without digitalization, we will be disconnected from progress in just a few years. But even the most advanced digitalization will not be able to offer the expected solutions without involving politics and health research. Professional communication increases the willingness to transform. The preliminary scientific achievements described below represent examples of the necessary communication and division of tasks.
Advance scientific work: Difference between clinical research and healthcare research
Methodological aspects
The main difference between these two research directions is the question that needs to be answered and the person who makes the decision to allocate the intervention. In clinical research (KF), which predominantly uses interventional or experimental or explanatory studies, e.g. the Randomized Controlled Trial (RCT), the scientist makes the allocation decision and uses the random principle with the aim of causal relationships to describe, ie to provide proof of the principle (5, 6).
In health services research (VF), on the other hand, which predominantly uses non-interventional or pragmatic or observational (descriptive) studies – such as the Pragmatic Controlled Trial (PCT) – the practitioner makes the allocation decision to the best of his knowledge and belief. The scientist compares the decisions made by the practitioners and their results in everyday care (5, 6). However, if the scientist were to intervene in the care decision under everyday conditions by making a decision based on the random principle, it would hardly be justifiable to designate this framework as VF and to accept it for ethical reasons.
Continuum instead of dichotomy - a scientific misdevelopment
The majority of all users of the term "pragmatic trial" obviously assume that effects that can be observed under everyday conditions, eg the influence of various interventions on the quality of life of the patients, can be measured in a randomized study. This apparent contradiction may go largely unnoticed, even though the term “pragmatic trial” suggests the exact opposite of “experimental trial.” The reason: In a pragmatic study, everyday effects are observed, while in a randomized, experimental study the effects of an experiment are examined. In an analysis of 100 unselected studies in which health-related quality of life (hrQoL) was determined as the endpoint of an intervention, almost all of these studies (91%) were carried out as RCTs (7). A plausible reason for this decision is that no other recognized study design for comparative analyzes has yet been established. The fact that randomization is an unsuitable method for describing everyday effects has been well described (5, 6, 8), but is not yet implemented in research.
This initial situation must be taken into account when a group of well-known scientists assumes a smooth transition, ie a continuum between explanatory and pragmatic trials (9, 10).
The assumption of a smooth transition (continuum) is based on a non-experimental, inductive analysis from which a misinterpretation was derived. This misinterpretation has become an unchallenged statement through repeated reporting in influential journals (11, 12).
Likewise, it has not yet been discussed that the confirmation of this incorrect interpretation will have significant consequences for the development of AF. Clear differences between KF and VF were described by a working group of the German Medical Association (13). Without assuming a dichotomous distinction between KF and VF, this differentiation is not possible because both research directions (both KF and VF)
- answer different questions that are just below
- can be tested under different conditions and for what purpose
- different instruments are required.
A careful analysis of the effects that can be demonstrated by AF leads to a plausible but equally sobering result: it is almost exclusively our preconceived notions that block the development of AF (14). This view comes close to Klose's assessment, who discussed “want” in the context of digitalization. Precisely for this reason, it should be examined more carefully than before whether trend-setting decisions such as the transitions between explanatory and pragmatic trials (continuum) postulated by the PRECIS group (15) are based on a systematic error in the formulation of the initial hypothesis and the interpretation of the design of the analyzed studies .
Different opinions on the analysis concept
As a member of the PRECIS working group in 2012/13 (16, 17, incorrectly quoted under “Portzsolt”), Porzsolt pointed out a systematic error in the PRECIS group's concept. However, the alternative suggestion of starting with a hypothesis that assumes a continuum between explanatory and pragmatic trials and then testing this hypothesis based on collected data (deductive method) was obviously not acceptable to a majority. The PRECIS group analyzed a large number of available studies in order to derive statements about the possible continuum between explanatory and pragmatic trials from the data obtained (inductive method).
It was correctly observed that the majority of the published studies corresponded neither clearly to an explanatory trial nor clearly to a pragmatic trial. From this observation, the conclusion was derived that there is a smooth transition between explanatory and pragmatic trials, which is mapped in nine dimensions in the PRECIS wheel (15).
In our opinion, this conclusion is incorrect because it cannot be ruled out that the majority of the published studies analyzed did not or could not observe the defined criteria of an explanatory trial for two reasons:
• The first possible reason for the deviations is an unclearly defined study aim, because in many studies analyzed it was obviously not clearly defined whether a question about CF or VF, ie an explanatory or a pragmatic trial, should be carried out. The classification as one of the study types or as a mixed form was carried out retrospectively in all cases.
• The second reason is that there was already an international consensus on conducting explanatory trails, ie RCTs, but not on conducting pragmatic trials. The authors of the studies analyzed by the PRECIS group therefore had no choice but to either conduct a study strictly according to the definitions of an explanatory trial in order to answer questions of AF, or to answer other questions more or less from the international consensus to deviate.
We therefore assume that the now widely accepted concept of a continuum between explanatory and pragmatic trials is based on an incorrect conclusion. The assumed connection between explanatory and pragmatic trails was tested using an inductive rather than a deductive procedure. Inductive methods are controversial because they do not strictly follow the laws of logic. If a hypothesis had been tested first, this misinterpretation could very likely have been avoided.
There is a need for action because this misinterpretation has significant effects on the future development of VF: It blocks VF because it contains the requirement that all pragmatic studies be randomized. As long as this unfortunate association of “pragmatic and randomized” persists in the literature, a distinction between KF and VF will not be possible on a methodological level.
An additional error when updating the CONSORT statement
The 2008 update of the original CONSORT statement from 2001 describes the aim of expanding the scope of the CONSORT statement from initially only randomized studies to include pragmatic studies (18). However, this expansion was not implemented by expanding the statement, but simply by renaming “pragmatic trials” to “randomized pragmatic trials” (18). This renaming was justified by the absolutely clear and frequently cited publication by Schwartz and Lellouch from 1967 (5). However, the word “randomized” is neither mentioned in this publication nor is there a statement to be found there that can be used to justify the renaming described. Therefore, in the interest of the VF, it should be discussed to propose a new update of the CONSORT statement in order to initiate a further development of the accepted terminology.
The asymmetry of the evaluation of “explanatory” and “pragmatic trials”
There is a consensus that only patients who meet seven criteria are included in explanatory or interventional trials:
- There must be a formal study protocol approved by an ethics committee.
- Liability insurance must be taken out to carry out a human experiment.
- Participating physicians must demonstrate experience in conducting studies.
- The patient must meet all inclusion criteria.
- The patient must not meet any of the exclusion criteria.
- The patient must agree to the randomization.
- The patient has to sign an informed consent.
Nevertheless, it cannot be avoided that a doctor will not include all eligible patients in an RCT if he expects that problems with patient consent, compliance, tolerability or the response to the therapy will arise. Every practitioner is familiar with these problems, but not every scientist, because this information does not always find its way from healthcare to research. This “unnoticed patient selection” makes it difficult to transfer the results of RCTs into everyday care (19).
In contrast to an RCT, however, all patients who meet the inclusion criteria and are cared for in a PCT at a participating institution (practice or clinic) during the defined recruitment interval are to be included in a PCT. The inclusion criteria of a PCT are intended to describe the spectrum of patients treated in everyday life, ie under Real World Conditions (RWC).
Exclusion criteria do not exist under everyday conditions because all patients have to be cared for under RWC. If these criteria are required in future PCTs, it can be confirmed that the care of the far larger proportion of all patients takes place under RWC. In these patients, however, neither the endpoint-specific initial risks nor the therapies or the results achieved can be systematically recorded. In other words: apart from a few exceptions, such as exemplary standardized studies in malignant hematological diseases, there are no standardized data that allow unbiased comparative analyzes of everyday care.
Summary and discussion
In this first part of our two-part article, the opportunities that arise from digitalization can be identified from the perspectives of politics and science. Through the discussion about the Federal Ministry's roadmap for the development of the Digitalization Act, the need to also talk about the hurdles that have to be overcome in order to connect technical advances in telecommunications with everyday supply was recognized. Without this connection, it will hardly be possible to get the added value from digitalization that the system actors want. It has been recognized that the development of healthcare is lagging behind technological progress. The consequence that can be derived for health research describes the backlog that the VF now has to make. This contribution could influence the international discussion on pragmatic trials by identifying the problems arising from current assumptions. The RCT is undisputedly the gold standard in KF. However, it is unlikely that the same method will be the gold standard in VF, which has different goals than KF.
Our article specifically addresses the previously unresolved problems, whose undesirable effects on VF have so far been too little noticed and discussed. We therefore see the first part of our article as a suggestion from the scientific perspective of the VF to the political perspective of the VF. Science depends on cooperation with projects that deal with the regulations that can facilitate the implementation of the VF.
The Bayes theorem described in the 18th century (20) and new non-experimental strategies used in observational studies outside of medicine are suitable for establishing causal relationships (21). This is the direction in which health research will also develop. This assumption is supported by the successful application of Bayes' theorem in World War II to decipher the German Wehrmacht's secret ENIGMA code and the exact localization of Russian submarines (20). When it comes to proving causal relationships, we have made mistakes so far by relying on unsuitable methods or interpreting their results incorrectly. If lack of transparency, lack of reproducibility, violations of validity and errors in interpretation are recognized and avoided, valuable information can be gained (21).
In the second part of our report, we will discuss the details of the necessary framework and the method of Pragmatic Controlled Trials (PCT), which relies on very similar strategies as Bayes' theorem for proving causal relationships. (22, 23). This strategy is proposed to collect interpretable data from everyday care. From the perspective of scientific evaluation, everyday care corresponds to a “natural chaos”, which must first be structured in order to be able to extract reliable data. The proposed Pragmatic Controlled Trial (PCT) method is presented in Part II on the connection between digitalization and health services research. <<