Evidence-based quality measurement as a prerequisite for value-based healthcare

The quality assessment of medical care based on quality indicators plays an important role in the German health care system. The quality dimensions "Structure, process and result quality" developed by Donabedian in 1966 represent an established classification for quality indicators (1). Another way of classifying quality indicators is, for example, the throughput model according to Pfaff (2), which differentiates between input, throughput, output and outcome of the respective measurement objectives. In this context, measurement targets represent the event of the quality indicator to be measured. “Quality indicator” is not a protected term and is used by different actors in the healthcare system for a variety of issues. In inpatient care in particular, quality indicators are used to measure and depict the quality of care (3,4). Quality indicators are also increasingly used in the outpatient area (5-7) and across sector boundaries (8) to measure the effect of healthcare.

>> Until a few years ago, quality indicators were used for internal analysis and evaluation, primarily by the medical profession themselves, and thus made an important contribution to the internal quality development of a service provider. The consistent publication of quality indicators, including for patients, was neither planned nor wanted in the early years of the development of external inpatient quality assurance in accordance with Section 137 SGB V, for example. Internal quality management was promoted by comparing quality indicators in a “protected space”. The aim was to control and further develop the treatment results, particularly by means of structural and process indicators, and in many cases this was achieved (9). The aim was to promote quality development among service providers. In this context, the regular and structured publication of quality results in the sense of internal reporting is necessary in order to provide continuous feedback on the treatment results. The importance and importance of result indicators should not be underestimated, even if the opinion is often expressed that the development of result indicators is too demanding (10). Result indicators are necessary for a comprehensive presentation of results in order to provide an overview of the short-term (e.g.: hospital mortality) and medium-term (e.g.: readmission rate up to one year) results of the treatment carried out.
As part of the current further development of existing quality assurance procedures, patient-reported results are certified to be highly relevant and methods for recording them are integrated (11,12). A distinction is made between PROM (patient reported outcome measures), such as pain intensity (13) and PREM (patient reported experience measures), which include, among other things, patient satisfaction with the preparation for inpatient discharge (14).

Current situation of development and
Assessment of quality indicators
There are systematic and unsystematic approaches for the development of quality indicators (15-17). Unsystematic approaches are often pursued when data is available from which information for quality indicators can be obtained. Even though these indicators offer the advantage of being able to display results in a timely manner, there is little information regarding their quality. Systematic approaches to developing quality indicators are preferable because they help to increase the validity of quality measurement (18). Components of a systematic approach include (i) the description of the measurement objective, (ii) the selection and/or development of a preliminary set of quality indicators, (iii) the methodical evaluation through the consensus of the experts involved using structured procedures, (iv) the empirical one Testing the agreed indicators and (v) preparing the results report with subsequent use of the indicators in routine operations (19).
There are different instruments for the methodical assessment of quality indicators (20). The Institute for Quality and Patient Safety (BQS Institute) developed the QUALIFY assessment tool in 2007. This includes 20 criteria that are assigned to the areas of “relevance,” “scientificity,” and “practicality” (Table 1) (21). The AQUA Institute, which is currently developing quality indicators on behalf of the Federal Joint Committee (GBA) for external inpatient quality assurance in accordance with Section 137 SGB V, uses the quality criteria relevance, comprehensibility, practicality, (collection) effort, risk adjustment, and ability to discriminate to evaluate quality indicators and suitability for public reporting (20). This selection is based on the internationally proven RAND/UCLA Appropriateness Methodology (RAM), which was developed by the RAND Corporation and the University of California Los Angeles (UCLA) in the late 1980s (22). In addition to these instruments, there are other methods for evaluating quality indicators (23). In general, these instruments or procedures modified from them are used for the methodical assessment of quality indicators within the framework of structured Delphi procedures. In the Delphi process, written surveys of named experts are carried out in several rounds of queries on defined questions. This option has proven to be extremely practical and promising in the derivation and definition of quality indicators (24).

Regardless of the type of development, the quality dimension or the reporting type of quality indicators, the goal formulation, ie the purpose of the quality measurement, is crucial. According to Mansky (25), measuring the quality of treatment provided can pursue various goals:
• Communication between doctors and nursing staff for a collegial exchange about the quality of care
• Quality development (performance) among service providers / within a hospital
• Informing the public (e.g. benchmarking)
• Basis for compensation
Requirements for the quality of
Quality indicators
From a methodological point of view, quality indicators are instruments for measuring the quality of care under everyday conditions. In addition to the various target formulations listed, it must be clarified how the quality of treatment is defined at the levels of structures, processes and results depending on different patient groups, forms of treatment and diseases. According to the authors, quality indicators per se must meet the same high standards as those placed on endpoints or target parameters in clinical studies (26), that is, they must be valid, reliable and practicable (27). Validity means that a quality indicator completely and correctly measures the construct being measured. Reliability includes the extent to which multiple measurements of a quality indicator on the same population agree. In addition to these classic quality criteria, a high level of practicality is also essential for the implementation and acceptance of measuring instruments in routine operation (26,28). Practicality includes, on the one hand, the possibility of data availability and collection and, on the other hand, the interpretability of the quality results (29). Since, in contrast to clinical studies, no randomization takes place in routine care, adequate risk adjustment is also necessary, at least for indicators of the quality of results.
If one differentiates between quality indicators based on their target formulation (25), quality indicators that are used exclusively within an institution may have less stringent requirements (validity, reliability, practicability) than quality indicators that are used for public reporting, in particular for the purpose of benchmarking ( Validity, reliability, practicality, risk adjustment (21,30-32)) can be used. This discussion is currently taking place internationally. (33,34).
Status quo of quality measurement in
German healthcare system
The Quality Indicator Thesaurus of the National Association of Statutory Health Insurance Funds (QUINTH) provides an overview of German-language quality indicators (35). This database was created by the BQS Institute and includes 1,667 quality indicators (as of October 30, 2014). For all quality indicators, information on their area of application and data basis, quality and their suitability for Pay for Performance is available in QUINTH, which is why this database is a valuable resource. Using various search filters, such as indicator type (structure, process or result quality), status or degree of implementation, the user of this database has the opportunity to research the appropriate quality indicator for their area of application and purpose. Process indicators (n=724; 43.4%) currently form the largest group of indicators listed in QUINTH. Quality indicators for measuring structure (n=384; 23.1%) and result quality (n=559; 33.5%) are less frequently represented. In QUINTH, outcome quality does not only refer to treatment results, but also includes areas that do not concern direct patient treatment, such as “doctors who feel well informed about goals and activities in the care model” (36). Looking at the reported evidence of the quality indicators presented, 112 quality indicators (6.7%) are certified by their authors as having a high level of evidence, although only 55 (3.3% of all quality indicators) have “high” validity (Figure 1). The analysis of the reliability of the existing quality indicators shows a similar picture. Of the 112 quality indicators with “high evidence,” 34 were reported to have “high” reliability (Figure 2). A total of 34 / 1,667 quality indicators are shown to be valid and reliable (2% of all indicators contained in QUINTH). It is not clear to the user of QUINTH which methods the authors of the quality indicators use to assess their evidence and which data justify the various classifications (low, medium, high). This means that the methodological quality of even the “best” 34 quality indicators remains unclear to the user.
A systematic review of all current S3 guidelines shows a similar picture (as of November 30, 2013). Guidelines are systematically developed basis for decision-making for diagnostic and therapeutic measures (37). S3 guidelines represent the gold standard in Germany and are based on systematic evidence research, assessment and synthesis as well as a systematic consensus process. As part of the systematic review, 87 current and valid S3 guidelines were identified, 34 of which contained defined quality indicators (38). A total of 344 process indicators, 17 structural and 33 result indicators were defined. None of the guidelines made any statements regarding the quality of the quality indicators.
Interim conclusion
The analysis of the S3 guidelines and the findings from the QUINTH database confirm the current situation of quality measurement in Germany: There are a lot of instruments (quality indicators), the quality and therefore suitability of which - at least for the purpose of public reporting and quality-related remuneration - is largely the case is unclear. We know from clinical research and health services research that measurement instruments of unclear quality can overestimate, underestimate or completely misjudge the effects of interventions and therefore their use must be critically questioned (39).
Learnings regarding quality indicators from evidence-based medicine
Due to the high heterogeneity of the measurement instruments used to represent the treatment effects in clinical studies, the comparability of the results and therefore the pooling of study results based on a meta-analysis as part of systematic reviews is often only possible to a limited extent or not at all. As a result, the translation of the knowledge generated in clinical studies into practical clinical care is significantly hindered (40). The problem “Clinical trials are only as credible as their endpoints” (41) has long been known in medicine. In this context, the Cochrane Collaboration plays an essential role in bridging the gap between medical knowledge and medical practice by summarizing the quality of clinical studies, critically assessing them and making the available evidence widely accessible. One of the four main objectives of the “Cochrane Strategy 2020” is to inform decision-makers in order to sensitize them as those responsible for evidence-based healthcare (42). However, this can only be achieved if relevant and consistent study endpoints are measured and compared to determine the effectiveness of medical measures. Against this background, the development of so-called Core Outcome Sets (COS) was initially initiated in 1993 by the Outcomes Measures in Rheumatology Initiative (OMERACT) (41). The aim of OMERACT is to recommend disease-specific agreed outcome domains (core set consisting of outcome domains) and then corresponding valid, reliable and practical measurement instruments (core set consisting of outcome measurement instruments) for each outcome domain, which are measured in every clinical study should. Of course, in addition to the COS, other target criteria can be used to meet specific study requirements or questions. Following the example of the OMERACT Initiative, COS for clinical studies have been developed in recent years for a wide variety of medical areas (diseases as well as interventions). A summary of current COS can be viewed via the database of the COMET Initiative, which aims to promote the development, reporting and implementation of COS in clinical studies (43). In a systematic review conducted in 2014, 198 COS were identified (44).
In order to obtain meaningful measurement results, high demands must be placed on measuring instruments and their psychometric test properties. The international COSMIN group (COnsensusbased Standards for the selection of health Measurement INstruments) conducted a multiprofessional Delphi study to define and agree on the terminology of quality requirements and their content (29). The focus of this effort was on patient-reported instruments (PROM). Nevertheless, the selected requirements also apply to measuring instruments used by doctors or nursing staff.
A comparable methodology is fundamentally proposed by the authors in order to develop standardized, relevant and universally interpretable quality indicators.
Evidence-based quality measurement as a prerequisite for value-based healthcare
The evaluation principle of the entire care process based on adequate, i.e. valid, reliable and patient-relevant outcomes with quality indicators is a central component of Value-based Healthcare (VbHc) (45). VbHc aims to maximize patient benefit per euro spent (Fig. 3). What is crucial here is the condition-specific, multidimensional measurement and evaluation of the actual treatment results, since process quality alone cannot cover all possible interventions and capture the large number of patients with heterogeneous initial conditions and risk factors. Based on these criteria, the appropriate diagnostics and therapy are selected as part of the treatment process between the doctor and the patient, taking into account the individual patient preferences. All provided that the necessary structural features, such as the qualifications of the staff or the medical technical equipment, are available for optimal treatment (46,47).
The focus of VbHc is the process of disease prevention, detection, treatment and monitoring of follow-up from the patient's perspective in the full cycle of care. The analysis of the quality of treatment at the level of a service provider has a subordinate role, which only serves to determine the partial benefit for the patient. All service providers involved in the treatment process have the responsibility to work together to achieve the best possible outcome. This is a common principle, especially in health networks, to ensure high-quality medical care across the board, regardless of which network partner the actual service is provided.
In order to measure and map treatment results, the development, selection and application of adequate quality indicators is necessary. These are predictors of later treatment results. PROM contributes to this as an essential component of comprehensive measurement from the patient perspective. These enable the assessment of essential quality indicators and treatment results from the patient's perspective. However, they also provide the opportunity to retrospectively measure patient compliance during the treatment process. This means that the results of the treatment can be analyzed and evaluated from different perspectives.
According to Porter (45), the founder of VbHc, there is no alternative to measuring the actual treatment results and their costs at the patient level, since the goal of healthcare is to maximize patient benefit (=value).
outlook
The measurement of relevant study and treatment endpoints is an essential part of evidence- and value-based healthcare. COS are being used more and more frequently in clinical studies in order to be able to comparatively assess the benefits of an intervention based on the endpoints of clinical studies. In routine care, quality indicators have so far been used primarily for analysis and evaluation in the medical profession or to improve the quality of service providers and thus also defined related requirements. With the change in health policy intentions away from exclusively internal use and reporting of quality results towards their public presentation at the level of service providers and the associated quality-oriented remuneration, these requirements are changing. Quality indicators must meet high (quality) requirements in order to be suitable for both internal quality improvement and public reporting. What is needed here is systematic and structured support to implement an assessment based on treatment endpoints across structural boundaries. Against this background, in our opinion, scientific research should be carried out in an evidence-based consensus process involving all relevant interest groups.
ed measuring instruments are developed as a core set for recording, evaluating and also communicating the quality of medical treatment. <<