Statistical risk models: possible applications for optimizing patient care

Given the challenges of demographic change, the healthcare system is faced with the question of how to allocate resources efficiently and fairly.
Only 20% of the insured are responsible for 81.2% of the hospital, medication and doctor expenses of the statutory health insurance (GKV) (1). Approximately 5% of patients account for 50% of utilization (Fig. 1). Almost half of these expenses relate to hospital treatment. In many cases, these are unavoidable and serve to ensure the safety and good quality of treatment for patients. A common reason for claiming benefits is the presence of a chronic illness. The top 10 reasons for hospitalization include diagnoses such as heart failure, angina, hypertension, and type 2 diabetes (1). The acute deterioration of chronic diseases such as diabetes or heart failure often leads to hospitalization. The risk of hospitalization increases significantly if a person has several chronic illnesses at the same time. In order to be able to assess a person's risk, risk prediction models are developed (predictive modeling). The aim of risk models is to avoid (re-)admissions to hospital, to prevent the incurrence of hospital costs and to achieve better health outcomes for the patient with intensive, cheaper medical intervention (2). Risk models can take a lot of information from the service sectors and master data into account in parallel and can be used to predict hospital stays for any reason as well as for specific diseases.

>> In Great Britain and the USA, the use of prediction models in healthcare is therefore widespread, particularly for predicting hospital readmissions (3). The NHS (National Health Service) in Great Britain has developed a model for the risk of readmissions (PARR - Patients at Risk for Rehospitalization), which is intended to enable stratification of risk populations for special forms of care (3).
What is special about this model is, on the one hand, the focus on readmissions - so initial admissions are not taken into account - and, on the other hand, the use of medical findings data. This takes advantage of the fact that the people under consideration have already been treated as inpatients and therefore clinical data from previous hospital stays is available.
In Germany, various methods for creating prediction models have been tested and developed, for example by Wolik on data mining methods (4), but these models have so far hardly been used in practice for prevention or care management.
American models were also adapted to German routine data, such as the Case Smart Suite, which was offered by DxCG/Verisk Health as CSSG (Case Smart Suite Germany).
Internationally, disease and case management projects have been initiated that are intended to delay or prevent the progression of illnesses or, depending on the illness, to accelerate healing. The results of the evaluations of these programs are very heterogeneous, but patients who are at high risk of using services tend to particularly benefit from such offers (5).
Risk prediction models to determine the probability of hospital admission can be developed using health insurance data.
Prediction-supported case selection for care management measures is carried out using the individually predicted Likelihood of Hospitalization (LOH).
Other models, such as high-cost models, have proven to be inefficient in the approach to reducing utilization, as high-cost cases usually arise from expensive illnesses such as blood clotting disorders or expensive therapies such as the administration of blood clotting agents or from acute events such as multiple traumatizations and are therefore often unavoidable.
Against this background, the AOK Scientific Institute (WIdO) has developed a prediction model for hospital stays that refers to patient care in Germany, which makes both initial and readmissions the subject of the prediction, based on a very large data set of AOK insured people was developed and essentially evaluates routine performance data from a health insurance company.
New prediction model for
Hospital admissions
The WIdO risk prediction model for calculating the individual probability of hospital admission was created on a random sample of 12 million anonymized AOK insured people who have been insured continuously for three years. Billing data from outpatient and inpatient care, medications and master data information such as age, gender and level of care are taken into account.
Classification systems were developed for outpatient and inpatient diagnoses (based on ICD), for drugs (based on ATC) and others such as regional factors or classifications into acute and chronic diseases.
The influence of patient dispositions and their previous use of health services on the risk of hospital treatment was examined using logistic regression.
The prediction model developed assigns each insured person's personal probability of being admitted to hospital in the next twelve months (initial admission and readmission). The individual likelihood of hospitalization is between 0 and 1, i.e. between 0-100 percent.
The quality criteria variance explanation, sensitivity and the positive predictive value PPV (positive predictive value also takes the false positive rate into account) achieve good values for the general model: the R-squared is 0.20, the sensitivity is 0.14 and the PPV is 0.58. A PPV of 0.58 indicates that 58% of predicted hospital admissions occurred.
The ALOH-A 2.0 model developed by WIdO and the CSSG-AOK 0.8 from Verisk Health achieve high quality criteria, with the more current model ALOH-A 2.0 achieving higher values (Table 1).
As expected, in relation to morbidity groups (heart failure, diabetes, CHD, asthma, COPD and depression), there are particularly higher sensitivities and variance explanations, which are further increased when evaluating the top quintile of predicted values. The increase in sensitivity is due to the fact that more homogeneous insured people with higher hospitalization rates are brought together in morbidity groups.
The overall data set includes both insured people who have not had any claims in the previous 2 years as well as seriously ill people. The subpopulations of the morbidity groups only include people who were being treated for their chronic illness during the analysis period. For heart failure, the sensitivity is 0.39 and the PPV is 0.59. The average predicted probability of hospitalization for heart failure patients is 40 percent; odds ratios for the influencing factors were determined in general and specifically for morbidity groups.
When selecting the variables taken into account in the regression, quality criteria and significance were the deciding factors. Variables that could not provide additional explanation of the variance were gradually eliminated. Other information that could be prepared using routine data and publicly available information was tested for effectiveness. Only information that helped explain the variance remained in the model. For example, no effect of the level of coverage in a region on the LOH was found to improve the model.
Use in case management
The CSSG 0.8-AOK, which was jointly developed by WIdO and Verisk Health, was used by the AOK Baden-Württemberg and the University of Heidelberg from 2009 onwards in the PraCMan project (“family practice-based case management for chronically ill patients with a high risk of hospitalization”). This model uses proprietary classification systems for medical diagnoses and medication administration developed by Verisk Health.
This LOH model has already been used in a pilot study for case selection for general practice-based case management (6). This showed that multimorbid older patients in particular were identified who were potentially suitable for case management according to clinical criteria (7). In addition, it was shown that filter formation plays an important role because not all high-risk patients are equally willing and able to actively participate in an intensified care model. The participating general practitioners also overwhelmingly rated the offer of pre-selection of insured persons using a predictive model positively. As part of an evaluation study on general practice-based case management for multimorbid patients (PraCMan), a combined case selection was carried out using the CSSG-AOK and clinical assessment by doctors (8); an introduction to the quality of the CSSG model will be provided in a later article (in preparation).
Practical use
The independent WIdO model version 2.0 was completed in December 2011. Initial indications suggest that its use is efficient. To use the WIdO model, routine data from 8 quarters is required; the insured persons are then limited via the definition of care management, which specifies which illnesses insured persons should be included and how they should be contacted (e.g. via the family doctor). The disease can be selected based on disease frequencies, and the pilot region can be selected based on the localization of increased general or indication-specific LOH average values. In order to be able to react quickly to current changes in patients, especially to worsening health conditions, the forecast values should be updated quarterly.
In addition to calculating the probability of hospital admission, a filter is created that defines inclusion and exclusion criteria. In collaboration with medical professionals, the exact definition of the target disease of the intervention is specified, for example whether only type 2 or also type 1 diabetics should be addressed and which other diseases must or may not be present. A common exclusion criterion for interventions that have a high level of interaction directly with the patient is, for example, the presence of dementia. Nevertheless, it makes sense to deviate from the general selection criteria in individual case decisions on site.
An evaluation of the practical application of the prediction model requires a long period of time from the selection of the insured to the approach, the start and end of the intervention and a follow-up period of at least one year.
The minimum follow-up period is relatively long because some interventions, such as improving the medication management of heart failure patients in accordance with guidelines, initially lead to an increase in utilization costs due to the increasing number of prescribed medications and it takes time until expensive hospital admissions are avoided.
An evaluation of the model is not possible at this time.
Compliance with data protection is guaranteed in various places.
When creating models, the WIdO works exclusively with anonymized data; there is no general screening in the AOKen (in Baden-Württemberg the prediction models were only used for HzV patients). Data protection officers on site in the AOKen and in studies in the participants' committees (such as the ethics committee) decide on the procedure.
discussion
With the LOH model presented, a valid prediction model for future hospital admissions based on performance data from a health insurance company was developed on the methodological basis of multiple logistic regression.
A comparison with other available models showed higher or good quality criteria (as far as comparable).
The PPV value is more important than the sensitivity (true positive rate) for evaluating the suitability of the model in care management. If mammography screening is taken as an example, breast cancer is detected in 4 out of 5 women, which corresponds to a sensitivity of 80%. At the same time, significantly more women are suspected of having breast cancer who do not have breast cancer. Mammography screening has a poor PPV because more findings are false positives than true positives (four-field table).
This correction is important when predicting hospitalization because a precautionary overestimation of the LOH value improves sensitivity but leads to more false predictions of hospitalization. The PPV specificity indicates what proportion of positive predictions are truly positive and is therefore considered more important than sensitivity in this context.
The preventive application of the WIdO model and the transfer of the LOH prediction values into interventions are currently being developed in individual applications.
In principle, the prediction of future hospital admissions has some methodological limitations, especially since not all hospital stays are predicted: someone who suffers a heart attack may have previously had health problems, may have consulted a doctor, but does not have to have had any symptoms before or if they had them had to consult a doctor. In addition, the prediction models using routine data lack medical parameters and behavioral and person-specific risk factors for diseases that are considered to be a decisive factor for hospital admission for certain diseases. Nevertheless, entering 488 variables leads to a complex inclusion of a wide variety of possible combinations of these classifications.
These possible combinations far exceed the possibilities of manual or partially automated selection, since from the perspective of innovative care management, insured persons have a high risk of claiming benefits if this can be statistically derived from a combination of several characteristics. It is not necessarily the patients who have one or more chronic illnesses or who have had a high need for services in the past or who have been in hospital who are admitted to hospital in a short period of time (i.e. within a year).
Based on routine data, information is aggregated to determine who is likely to be admitted to hospital. The outlook for the application of the prediction model designed for use in the AOK system aims to avoid hospital admissions. Preliminary tests and retrospective considerations have been completed and application is already underway on a small scale. The model is to be used across the board in a supply management intervention in a federal state in 2013. The subsequent evaluation of the model after the intervention and the follow-up time for the evaluation will provide important information about its effectiveness. <<