http://10.24945/MVF.04.17.1866-0533.2028
>> Who are these four researchers who criticize their own profession in the magazine “nature” (issue 02/2016)? They are David B. Allison, a professor in the Biostatistics Department at the University of Alabama at Birmingham, and three scientists in the Office of Energetics at the same university - Andrew W. Brown, Brandon J. George and Kathryn A. Kaiser. It all started when Allison read a research paper in a respected journal that sought to assess how changes in fast food consumption affected children's weight. He noticed that the analysis was based on a mathematical model that overestimated the effects by more than tenfold. Allison wrote to the author, and the paper was retracted by the authors a few months later.
Together with the other researchers, Allison then began to pay particular attention to serious errors in the research. If such errors occurred, his team contacted the publisher or the authors. The researchers encountered many hurdles, meaning that peer reviews could not be carried out consistently after the initial publication. The scientific community needs to improve in this regard, the researchers write in the “nature” commentary. Science relies too complacently on the principle of self-correction. At the same time, however, in the practice of scientific publishing there would be great resistance to such a correction and strong deterrents would be used, which are summarized below.
After discovering an error in a publication, the research team received original data that was posted online. It also received institutional permission to reanalyze the data. A notice was then sent to the publication's editor through the manuscript submission system, describing the need to make a correction within two weeks. The journal decided 11 months later to accept the research team's statement of concern and retract the article. However, at that time the journal had neither published the notice of concern nor announced the retraction of the article.
The research team had to decide for themselves whether they should send the letter of concern to a team member of the journal or the editor, submit the letter formally as a manuscript, or contact the author of the paper directly. In some cases, the decision was made to contact the authors if an error might only be an ambiguous description. The magazine was contacted in unambiguous cases. However, the magazines often do not provide the opportunity to reach the editors directly. Magazine employees often corresponded without identifying themselves or the editors.
When the research team discovered that a paper incorrectly stated that statistical adjustment would bias the result, they submitted a notice to the editor through the manuscript submission system. External statistical experts commissioned by the journal confirmed the error. The authors were asked to retract their articles, but they refused. Finally, the journal published the authors' response to the Allison team's communication and a summary of the reviewers' criticism. However, an editorial published by the journal distanced itself from responsibility, saying it was the responsibility of each individual author to ensure that statistical methods are correctly applied and valid.
After discovering an error, a journal offered the research team the following options: either post an online comment or pay a "discounted" fee of £1,716 for publishing the concerns. For another magazine from the same publisher, the fee was 1,470 pounds. The Allison team was outraged that research money would be spent on correcting other people's mistakes.
In at least two cases, the Allison team requested data from the authors but instead received only the summaries of the calculations or no data at all. One journal retracted a paper when its authors refused to show their data or acknowledge the discrepancies explain that the research team had identified and described in a communication to the journal. Direct collaboration with the authors proved to be a stumbling block in correcting the errors because the authors were unwilling to respond.
Allison's research team criticizes the fact that although online platforms such as PubMed Commons offer the opportunity to comment on published papers, journal editors are not involved as intermediaries. Furthermore, the comments would not be included in the literature. The concerns posted are rarely in prominent places on magazine websites and are not cross-referenced. According to the authors, this leads to readers believing that an incorrect paper is correct and based on it making incorrect decisions in their field.
Given their experiences and this unwillingness of science to self-correct, what do the researchers recommend? They note that the scientific journals have defined guidelines for the submission of scientific papers and peer reviews. And the “Committee on Publication Ethics” has already summarized corresponding recommendations for magazines. What is still missing, however, are “official guidelines” from recognized independent institutions for correcting errors. In their view, journals, publishers and scientific societies should standardize, streamline and publicize this process. Both authors and journals should also make data and code available quickly when questions arise. Researchers could also support this process by enlisting the expertise of other scientists in statistical analysis and experimental design. According to the authors of the commentary, anyone who discovers a potential problem in a study should get involved - either by contacting the author or editor, or by posting an online comment. In addition, the commitment of scientists in post-publication reviews should be recognized and encouraged with incentives. “An official study would also help determine whether our experiences reflect science as a whole and whether our recommendations are feasible and effective,” the researchers conclude. Robust science requires robust corrections. “It’s time to make this process less laborious.” <<
from English by Olga Gilbers
