http://doi.org/10.24945/MVF.03.24.1866-0533.2617

Full article as PDF
Full article as PDF (English - automatic translation)
Plain text as TXT

The WiZen case: methodological weaknesses? Schmitt: "Certification effect extremely likely." "Every study has limitations," Prof. Dr. med. Jochen Schmitt of the Center for Evidence-Based Health Care (ZEGV) and one of the main leaders of the WiZen study group (1) told "Monitor Gesundheitsforschung" (Monitor Health Services Research). Anyone who claims that health care studies exist without limitations has not understood the fundamentals of scientific work. Therefore, an important part of every scientific work is the discussion of the results, which includes a discussion of the methodological strengths, but also of any potential limitations, as was done extensively in the original publication of the WiZen study in the "Deutsches Ärzteblatt" (2). It is all the more surprising that Prof. Josef Hecken, impartial chairman of the Federal Joint Committee (G-BA) and chairman of the G-BA's Innovation Committee, addressed – to quote Hecken – "one or two methodological weaknesses" of the WiZen project on "Effectiveness of Care in Oncology Centers" (3), funded by the Innovation Fund, in his opening speech at the conference "The Innovation Fund: Between Sustainability and Further Development" (see p. 16 ff.). MVF does the fact-checking. Hecken's statement in his opening speech is primarily to be understood as purely political. He said this, Hecken stated, so that no one would later claim that he had pushed the project forward, when some people had claimed it had methodological problems. But which ones? Hanisch: "Some Points of Criticism" With these words, the G-BA head is primarily referring to a discussion article published in the "Deutsches Ärzteblatt" (DÄB) by Prof. Dr. Dr. Ernst Hanisch, long-time Medical Director and Chief Physician of the Department of General, Visceral, and Thoracic Surgery at the Asklepios Clinic in Langen, Hesse. He currently advises the Frankfurt law firm Dr. Buchert & Partner, whose clients include hospitals. However, Hanisch wrote his DÄB commentary, entitled "Some Points of Criticism," as a surgeon specializing in visceral surgery, surgical intensive care medicine, emergency medicine, and clinical acute and emergency medicine, and provided his email address as an associate professor at the University Hospital of Frankfurt am Main, stating that there was no conflict of interest. In his contribution to the discussion, as well as in the editorial entitled "Questioning Routine Data," published in 2016, Hanisch rightly pointed out the problematic nature of a study based on—in his words—"a secondary analysis of billing data (DRG system) that is primarily created for billing purposes (so-called routine data)." In his view, this "inevitably leads to the question of whether, with these non-validated AOK billing data, it is possible to achieve comparability between the two groups (certified versus non-certified) through risk adjustment using the Elixhauser score employed by the authors." He also believes that, in addition to survival time, "patient-reported outcome measures" should also be used to assess the quality of hospitals. Hanisch also criticizes the authors for addressing the issue of causality due to their study design, stating that it "seems unlikely that the association between initial treatment in a certified center and longer overall survival is not causal." The objection of a non-causal relationship nevertheless remains, and in his view, it could have been addressed, for example, with a propensity score analysis (or other "causal inference methods" such as "target trial emulation"), which could certainly achieve the potential of a randomized study. All of this sounds quite plausible at first glance, but it is not entirely consistent. In their concluding remarks on the discussion of the results of the WiZen study (6), the WiZen authors counter that SHI routine data are firmly established in Germany in health services research and quality assurance, including in the Federal Joint Committee (G-BA), and are the "best resource for generating evidence in the context of standard care." Furthermore, they do not use DRG data, as described by Hanisch, but rather cross-sectoral, audited routine data from statutory health insurance (GKV), which also form the basis of the quality assurance procedures according to Section 136 of the German Social Code (SGB V). Stang: "Evaluability of the Effect of Certification of Oncology Centers" In their closing remarks, the WiZen authors also address further discussion contributions, all published in the German Association of Oncology (DÄB). For example, the editorial by Prof. Dr. Andreas Stang from the Institute for Medical Informatics, Biometry and Epidemiology at Essen University Hospital entitled "Evaluability of the Effect of Certification of Oncology Centers" (7). In it, Stang warns of a "weak evidence base" and a "risk of systematic bias" and also states that the authors interpret their results "with great enthusiasm." Even though – according to Stang, who also states that there is no conflict of interest – the results appeared plausible, "the critical reader cannot fully understand the almost unqualified optimism associated with the authors' formulations such as 'very high robustness,' 'reliable and valid,' 'high representativeness,' and 'transferability is generally given.'" The WiZen authors counter that the current evidence in the area under study is not "thin." Numerous publications exist that consistently demonstrate benefits of initial treatment of cancer patients in certified centers, including the impact on symptom burden and quality of life. Furthermore, Stang's editorial does not address this evidence base, but instead cites a Cochrane review that examined other interventions in other countries and a different context, namely the practical effectiveness of an accreditation process to reduce MRSA infections in hospitals in South Africa and England. Fröhner: "Certification does not necessarily reduce mortality." The third DÄB editorial on the WiZen study was written by Prof. Dr. Michael Fröhner, who at the time of the editorial was still Chief Physician of the Department of Urology and has been Medical Director of the Zeisigwaldkliniken Bethanien Chemnitz since January 1, 2024. He points out that he serves as Chief Physician of a clinic run by a religious organization that is certified as a prostate cancer center. In his commentary entitled "Certification does not necessarily reduce mortality" (8), he opens a completely different can of worms. He writes that "analyses intended to justify such redistribution require caution in interpreting the data," which is especially true "when they are financed and conducted by interested parties." The WiZen authors were determined not to accept this accusation. In their closing remarks (7), they point out that the study was funded by the Innovation Fund and that it can be assumed that "all Innovation Committee members are interested in the continuous optimization of the healthcare system." Furthermore, the Innovation Committee assessed the study results "as evidence of a significant increase in survival rates through treatment at certified oncology centers" and classified the study as "groundbreaking" and "convincing." "Consequences of action and inaction are equal." In the authors' view, the results of the WiZen study provide strong evidence that all patients should receive initial treatment in certified hospitals, and very little evidence against it. From an ethical perspective, the WiZen authors continue, "the consequences of action and inaction are equally valid." They therefore recommend that "policymakers, self-administration, patients, and referring physicians consider these results of the WiZen study in their actions." Although the authors clearly state in the original publication of their work (2) that the study evidence to date "does not allow a conclusive assessment of the effects of treatment in certified hospitals/centers for individual entities," regional analyses indicate survival and economic benefits for treatment in certified hospitals. In an interview with "Monitor Versorgungsforschung," one of the lead authors, Prof. Dr. Jürgen Schmitt from the ZEGV (German Association of Cancer Research), goes a step further. He states quite clearly: "It is extremely likely that there is a relevant certification effect and that many cancer patients would benefit from centralization in certified centers."  

WiZen

1: Center for Evidence-Based Healthcare, Carl Gustav Carus Medical Faculty, TU Dresden, Dresden: Prof. Dr. med. Jochen Schmitt, Dr. rer. nat. Veronika Bierbaum, Christoph Bobeth, Dr. rer. pol. Martin Rößler, PD Dr. rer. nat. Olaf Schoffer / Tumor Center Regensburg, Center for Quality Assurance and Health Services Research at the University of Regensburg: Prof. Dr. med. Monika Klinkhammer-Schalke, Dr. med. Michael Gerken / Scientific Institute of the AOK, Berlin: Patrik Dröge, Thomas Ruhnke, Christian Günster / Association of German Tumor Centers eV, Berlin: Prof. Dr. med. Monika Klinkhammer-Schalke, Kees Kleihues-van Tol. 2: Schmitt J, Klinkhammer-Schalke M, Bierbaum V, Gerken M, Bobeth C, Rössler M, Dröge P, Ruhnke T, Günster C, Kleihues-van Tol K, Schoffer O, on behalf of the WiZen Study Group: Initial cancer treatment in certified versus non-certified hospitals: results of the WiZen comparative cohort study. Dtsch Arztebl Int 2023; 120. DOI: 10.3238/arztebl.m2023.0169 3: https://innovationsfonds.g-ba.de/projekte/versorgungsforschung/wizen-wirksamkeit-der-versorgung-in-onkologischen-zentren.137 4: https://www.aerzteblatt.de/archiv/238126/Einige-Kritikpunkte 5: Dtsch Arztebl Int 2016; 113: 251; DOI: 10.3238/arztebl.2016.0251b 6: DOI: 10.3238/arztebl.m2023.0265 7: Stang A: Evaluability of the effect of oncology center certification. Dtsch Arztebl Int 2023; 120: 645–6. DOI: 10.3238/arztebl.m2023.0184 8: Dtsch Arztebl Int 2024; 121: 202-4; DOI: 10.3238/arztebl.m2023.0264 9: Raspe H, Pfaff H, Härter M, Hart D, Koch-Gromus U, Schwartz FW, Siegrist J, Wittchen HU: Health services research in Germany: status – perspectives – funding. Statement. In: DFG Standpoints. Weinheim/Berlin: Wiley-VCH 2010; 1-50.