Country rankings based on OECD health data – an analysis of the methodological problems

Health statistics from the Organization for Economic Cooperation and Development (OECD) are often used to classify healthcare provision in Germany. Accordingly, the German case number rates for certain procedures are among the top internationally. These country rankings are repeatedly taken up in the specialist and daily press. A frequently cited example is hip replacement surgery, which, according to OECD statistics, is only carried out more frequently in Switzerland than in Germany (Fig. 1). Another question that is often discussed is the amount of health care spending. Here, the German healthcare system is considered expensive by international comparison, as shown by the OECD country ranking of healthcare spending per capita, in which Germany ranks 6th (Fig. 2). However, ranking using OECD data as a benchmark is fraught with a number of methodological problems. These problems result, on the one hand, from incomplete data in the reports to the OECD and from their lack of comparability. On the other hand, the OECD data generally do not take into account the influence of the different age structure in the individual countries, although the use of services and the level of health expenditure are age-dependent.

>> The number of surgical interventions in a country comparison was examined by the OECD in a study in 2013 (McPherson et al. 2013). McPherson et al. carry out direct age standardization - but only for selected interventions and countries. Only five procedures (cesarean section, hysterectomy, prostate removal, hip replacement and appendectomy) and 17 countries are analyzed based on the 2008 database. Among the selected procedures, prostate removal and hip surgery are only two procedures that focus on older people. For the remaining interventions, the focus is on younger people and age adjustment therefore has a small effect. The standardization based on the world population, which, with its focus on children, adolescents and young adults, is not suitable for a comparison of relatively old societies - especially in Europe - still appears to be problematic. The authors conclude from their results that age standardization of the OECD data would hardly influence the ranking of countries in a ranking.
This statement was taken up in another OECD publication by Kumar and Schönstein (Kumar/Schönstein 2013). Their study received media attention because it placed the German hospital case numbers in an international perspective. It has often been heard in the media and in the health policy discussion that Germany is the “surgery world champion” and occupies a leading position internationally in the number of certain procedures, such as hip or knee replacement operations.
The present study addresses the age dependency of the use of medical services and questions the statement by McPherson et al. The analysis shows how Germany ranks in both medical interventions and health spending in an international comparison if its relatively old population is adequately taken into account. In addition, methodological problems with regard to the OECD health statistics are highlighted, which also affect the comparability of the data.
Limits of comparability
The OECD imposes extensive requirements on member states in order to achieve uniform data reporting and thus comparability. Due to the diversity of health systems and national characteristics, complete uniformity cannot be achieved. Deviations from the OECD specifications have a direct impact on the comparability of the results and therefore also on the meaningfulness of country rankings. Among other things, the following problems can be identified when reporting data on case number and health expenditure statistics:
• Data for all interventions are not available in all countries. In international comparisons of surgical procedures, different numbers of countries are always included for each procedure. If a country has not provided current figures, the data from the last available year is used.
• Data reporting is not always based on routine administrative data, but also on results from (sometimes outdated) less reliable surveys.
• There are also incomplete reports for procedures that are carried out both on an outpatient and inpatient basis. Some countries only report inpatients because no outpatient data are available. In many countries, for example, B. Inguinal hernia surgery is usually carried out on an outpatient basis.
• In Denmark, Germany and Estonia, operations performed on an outpatient basis in a hospital are also included in the case numbers, although only inpatient hospital cases should be reported to the OECD.
• In some countries, data reporting does not include procedures performed in private hospitals. This means that the actual number of cases is underestimated.
• Currency problems arise when comparing health expenditure. The measure of health expenditure as a percentage of GDP also compares the gross domestic products.

In addition, the frequently used comparison of German spending with the OECD average suggests that the OECD average is the “correct” value. However, it should be noted that each OECD country is included in the average value with the same weight. The average is therefore also determined by very small countries such as Iceland. An Icelander weighs around 1,000 times the OECD average than a US citizen. It must also be taken into account that the OECD also includes countries such as Mexico and Chile, which are still noticeably removed from Western European standards, especially in the health sector.
Age standardization of the
OECD data – methodological approach
Indirect age standardization is carried out for the following 15 procedures:
• Use of a hip prosthesis
• Use of a knee prosthesis
• Removal of the appendix
• Removal of the palatine tonsils
• Removal of the gallbladder
• Inguinal hernia surgery
• Percutaneous transluminal coronary angioplasty
• Coronary bypass
• Transurethral removal of the prostate
• Open removal of the prostate
• Removal of the uterus
• Breast conserving surgery
• Total mastectomy
• Kidney transplant
• Stem cell transplant
The OECD data on health spending per capita are also age-adjusted.
Calculation of age-specific case numbers
The basis for age standardization is the German number of cases per procedure according to age and gender. These are included in the flat rate-related hospital statistics (DRG statistics) of the Statistical Office.
taken from the Federal Office for the reporting year 2012 (Federal Statistical Office 2013). The selection of interventions is made in accordance with OECD specifications (OECD 2014). In order to construct profiles from the values, the number of cases must be put in relation to the population of the corresponding age and gender. A suitable size is the number of people insured by statutory health insurance and private health insurance (including railway and postal workers and their relatives) as the group of people who use the health system.
The number of people with private health insurance is taken from the statistics on private compulsory long-term care insurance and the number of people insured with statutory health insurance is calculated from the risk structure equalization (RSA) data provided by the Federal Insurance Office (BVA). The basis is the RSA data reports from the individual health insurance companies (Federal Insurance Office 2013). The reported days of insurance per individual age are divided by 365.
The age-specific number of cases FZRi is determined by dividing the age-specific number of cases according to DRG statistics by the number of insured persons per age group. The FZRi expresses how many cases occur per 100,000 insured people in age group i in Germany:
This results in a profile for each procedure that shows the age-dependent frequency with which the procedure is carried out per 100,000 insured people. Figure 3 shows this using the example of hip replacement surgery.

Calculation of age-specific health expenditure
The German age-specific spending profiles are not directly available, but are calculated from the information on GKV and PKV insured people. In the first step, only the expenses in the GKV and PKV systems are included.
The average benefit expenditure of those insured by statutory health insurance by age is taken from the RSA data for 2012. The retirement expenditure profiles were based on the insurance-related data of all GKV insured persons and contain the benefit expenditure per insured day, differentiated by gender, age and main service area (Federal Insurance Office 2013). These per-day values are extrapolated over a year, so that average expenses per GKV insured person are available for each individual age.
The information on those insured with private health insurance is calculated from the head injury statistics of the private health insurance association for 2012. The head injury statistics show the average number of invoices submitted for reimbursement by an insured person in one year. This also includes services that the aid reimburses, as well as the deductibles recorded.
The values of the GKV and PKV insured persons per individual age and insured person affiliation are weighted by the number of insured persons and combined into an average value for the individual age groups. This results in an average profile for Germany (Fig. 4).
The corresponding formula for the average expenditure per capita (ApK) of 5 to 10 year olds is shown below as an example:
The German value of average health expenditure per capita resulting from this profile does not correspond to the value in the OECD statistics. The OECD data also includes health expenditure borne by the patient and the state. To take this into account, the calculated values are extrapolated using a factor so that the profiles are multiplied by the population of Germany to produce the value published by the OECD. This method assumes that the other types of expenditure contained in the OECD data are evenly distributed across age groups in percentage terms.

Determination of the values expected for the OECD countries when using German profiles
For indirect age standardization, the population numbers of the individual OECD countries, differentiated by age group and gender, from the 2012 revision of the United Nations World Population Prospects are used (United Nations 2013). By multiplying the German case number rates per capita of each age group i with the respective population number in the age group of the country to be compared, niLand, the expected number of cases for each country results from the addition across all age groups and in relation to 100,000 inhabitants (100,000 inhabitants/total number of persons in NLand).
Analogously, the expected health expenditure per capita is calculated by multiplying (weighting) the German health expenditure per capita ApKi for each age group i by the number of people in the same age group niCountry in the respective OECD country. The products are added and divided by the total number of people in NLand and multiplied by 100,000.
In this way, it can be determined for each OECD country what number of cases and health expenditure would be expected if the country in question had the German profiles with its given age and gender structure. These values must then be compared with the OECD values of the individual countries instead of the actual German OECD values.

Creation of a country ranking according to age standardization
The calculated expected values (application of German profiles) are compared to the values that actually occurred in the individual countries (OECD data). This results in the standardized case number rate ratio (SFV)1,
or the standardized health expenditure ratio (SGV)1
If the SFV or SGV is above 1, the number of cases/health expenditure in the OECD country under consideration is higher than in Germany. If the ratio is below 1, the number of cases/health expenditure is lower than in Germany. There would be no difference with an SFV or SGV of 1. These results can also be presented as a percentage deviation from the German values.
The OECD ranking of the unadjusted number of cases and health expenditure can be seen by what percentage the values of the other countries differ from the German ones. The number of cases or health expenditure is expressed in relation to the German value.
This unadjusted ranking is compared with a ranking adjusted through indirect standardization. The countries are ranked here according to the standardized case rate ratio (SFV) or the standardized health expenditure ratio (SGV). This makes it clear for each country what percentage deviation it has from the values expected with German profiles. This new ranking now shows Germany's position in a country comparison adjusted for age and gender effects.
Results
Using the example of hip replacement surgery, Figure 5 shows the comparison of the hypothetical number of cases per 100,000 inhabitants (which results from indirect age standardization based on the German age and gender profiles shown in Figure 4) and the raw OECD data.
Figure 5 can be interpreted as follows: The dark bar (“OECD data”) indicates the unadjusted number of cases per 100,000 inhabitants. The light bar (“Application of German profiles”) assumes that the German case numbers for hip operations by age and gender apply in the other countries. For example, according to OECD statistics, 250 hip operations are carried out per 100,000 inhabitants in Norway, which is less than in Germany. The age and gender standardized comparison value is only 223 per 100,000 inhabitants. This is actually the correct value for a comparison with the Norwegian number of cases according to the OECD. The standardized sample size ratio (250/223) is 1.12. This means that Norway performs 1.12 times more hip operations (and therefore 12 % more) than would be expected if German profiles were used. Table 1 shows the new country ranking using the example of hip replacement surgery, in which the number of cases is shown in percentage values for reasons of simplicity. For comparison, the old ranking with unadjusted OECD figures is shown on the left.
Due to indirect age standardization, Germany moves down from second place to fifth place. Switzerland remains in first place. However, its gap to Germany is increasing. In the OECD data, the Swiss case number is 6.8 % ahead of the German one. Taking into account the relatively young Swiss population makes it clear that in Switzerland - with 24.2 % more interventions compared to Germany - the age-adjusted gap is even clearer than the OECD data suggests. The age adjustment also means that Germany is overtaken by Austria, Norway and Luxembourg. In Norway, age-adjusted, 12 % more hip operations are carried out than in Germany - and not 13 % fewer as in the raw data. Table 3 shows the calculated age dependency for each intervention based on the German profiles.
If all interventions are sorted according to the size of the rank shift as a result of indirect age standardization, the picture shown in Table 4 results.2
The indirect age standardization changes the ranking of the countries for every procedure considered - with the exception of kidney transplantation. Germany is slipping backwards in the country rankings for all interventions that focus on old age. When it comes to procedures that are primarily carried out in young people (removal of the tonsils and appendectomy), the situation is the other way around and Germany is further ahead in the age-adjusted ranking than the raw OECD data would suggest.
A standardized ratio of OECD health expenditure (SVG) is also calculated for health expenditure compared to the reference value calculated using German profiles. As with the number of cases, a new country ranking can be created from this, in which the health expenditure ratio is shown in percentage values for reasons of simplicity (Table 2). The old ranking with unadjusted OECD figures is shown on the left.
Due to the indirect age standardization, Germany changes its place in the country ranking from 6 to 9. The USA, Norway, Switzerland, the Netherlands and Austria are still in the top 5 places, although their percentage gap to Germany has increased significantly. In the USA, age-standardized spending is around 107 % higher than in Germany, in Norway around 40 % higher and in Switzerland around 34 % higher. For Germany the value is 0 % because the German profiles serve as a reference and the resulting value is standardized to the OECD value. In addition, Germany is overtaken by Luxembourg, Denmark and Canada, which are behind Germany in the OECD ranking. When using the German profiles, the health expenditure of 8 countries is higher than that of Germany.
In summary, it can be stated that, in addition to the methodological problems described, the consideration of the age structure in the countries - unlike McPherson et al. and Kumar/Schönstein - certainly has an influence on country rankings in the health sector. Even if age standardization cannot solve all the problems of comparing countries, standardization is essential when interpreting the different number of cases and health expenditure between countries. <<