What is the Replication Crisis?

The replication crisis is an ever-present issue in scientific research. This article will look at this subject, providing a brief overview of this complex subject.

Science Research

Image Credit: Tartila/Shutterstock.com

The Replication Crisis: An Overview

The replication crisis, also known as the reproducibility crisis and the replicability crisis, is a crisis that impacts the methodology of scientific research. Over time, it has been realized by several bodies that the results of many scientific studies are hard or almost impossible to accurately reproduce. The reproducibility of empirical data is essential for the scientific method, so difficulties in reproducing the results of a study or theory undermine its credibility.

Because of this, valuable scientific knowledge may be lost. The peer-review process is an integral part of scientific research, and if the data cannot be reproduced it cannot be validated. The crisis particularly affects the medical and social sciences. Considerable efforts in these fields have been performed into revisiting classic results, confirming their viability, and assessing the reasons for their failure if they are found to be unreliable. There is strong evidence from survey data that the crisis impacts every natural science field.

Growing awareness of the problem led to the term being coined in the early 2010s. The identification of the issue and considerations of causes and remedies have led to the new scientific discipline of metascience which uses empirical research methods to examine empirical research practices.

Both obtaining and analyzing data are involved in empirical research. Because of this, there are two categories of consideration concerning reproducibility. Reproducibility is the validation of the analysis and the interpretation of the data. Repeating an experiment or study to obtain new independent data which verifies the results of the original study is called replication.

Causes

There are a few causes of low reproducibility. A major cause is publication bias, where studies can become statistically skewed in the chase for significant results and overwhelming the correct results. Additional causes include questionable data analysis practices such as researcher degrees of freedom, data dredging, and HARKing. There can also be a failure to adhere to good scientific practice due to the generation of unprecedented rates of new publications and data, and the desire to publish or perish.

Scope

The journal Nature highlighted the scope of the issue in 2016 with a poll of 1,500 scientists. 70% of respondents reported that they had failed to reproduce the results of at least one of their peer’s studies. 87% of chemists, 69% of physicists and engineers, 77% of biologists, 64% of environmental and earth scientists, 67% of medical researchers, and 62% of all other respondents reported this issue. 50% had failed to reproduce one of their own experiments.

Additionally, several respondents reported that academic editors and reviewers told them to tone down comparisons to original studies when they reported failed replications. Sometimes the problem appears to be deliberate, as well – out of respondents to a 2009 study, 2% admitted that they had falsified studies at least once, and 14% said they knew someone who did. According to one study, this misconduct is more prevalent in medical research.

A 2021 study found that papers with reproducible results tend to be cited less than papers with findings that cannot be replicated in leading journals. The authors of the study stated that one of the reasons for this is that the review team may face a trade-off between maintaining standards and how interesting the paper is. Additionally, the authors stated that some papers create hype, and there is pressure to receive grants and court media coverage.

Psychology: A Field at the Center of the Controversy

Psychology has a particular problem with the reproducibility of study data and results. A 2018 meta-analysis study of 200 papers concluded that psychological research, on average, is littered with examples of low statistical robustness. Social psychology has been particularly implicated, but the fields of educational research, clinical psychology, and developmental psychology all have problems with reproducibility.

A study published in Nature Human Behavior failed to replicate 13 out of 21 behavioral and social science papers published in Science and Nature, two leading journals. In another study, 186 researchers from 60 laboratories failed to replicate 14 out of 28 findings despite large sample sizes. One explanation is the prevalence of questionable research practices in the field of psychology, including selective reporting, partial publication of data, post-hoc storytelling, and optional stopping.

In a study, a majority of 2,000 psychologists who responded admitted to using at least one questionable research practice. Solutions are difficult to realize in the field, but large-scale collaboration between researchers to make their data freely accessible is becoming more common practice.

Is there a reproducibility crisis in science? - Matt Anticole

Medical Science

Non-reproducibility of results in the field of medical science can have serious consequences. The US Food and Drug Administration, for example, discovered that 10-20% of medical studies between the years 1977 and 1990 were flawed. In 2012, Begley and Ellis discovered that out of 53 pre-clinical cancer studies, only 11% could be successfully replicated.

A survey by Nature discovered that more than 70% of respondents had failed to reproduce the results of at least one other study. Over half could not replicate at least one of their experiments. John Ioannidis of Stanford University has called for widespread reform, with one solution being to build a more patient-centric model of medicine instead of the current practices which favor the needs of physicians, sponsors, and investigators.

The Future

For research to be valid, the results and data must be reproducible, and currently, there is a crisis in multiple scientific fields. There are no easy solutions to the replication crisis, but researchers, journals, academic editors, patients, and advocacy groups can play a role in ensuring that robust standards are adhered to in scientific research. With increasing focus on the replication crisis, there may be a way forward that will improve the validity and quality of research exponentially.

References:

Further Reading

Last Updated: Apr 13, 2022

Reginald Davey

Written by

Reginald Davey

Reg Davey is a freelance copywriter and editor based in Nottingham in the United Kingdom. Writing for News Medical represents the coming together of various interests and fields he has been interested and involved in over the years, including Microbiology, Biomedical Sciences, and Environmental Science.

Citations

Please use one of the following formats to cite this article in your essay, paper or report:

  • APA

    Davey, Reginald. (2022, April 13). What is the Replication Crisis?. News-Medical. Retrieved on May 25, 2022 from https://www.news-medical.net/life-sciences/What-is-the-Replication-Crisis.aspx.

  • MLA

    Davey, Reginald. "What is the Replication Crisis?". News-Medical. 25 May 2022. <https://www.news-medical.net/life-sciences/What-is-the-Replication-Crisis.aspx>.

  • Chicago

    Davey, Reginald. "What is the Replication Crisis?". News-Medical. https://www.news-medical.net/life-sciences/What-is-the-Replication-Crisis.aspx. (accessed May 25, 2022).

  • Harvard

    Davey, Reginald. 2022. What is the Replication Crisis?. News-Medical, viewed 25 May 2022, https://www.news-medical.net/life-sciences/What-is-the-Replication-Crisis.aspx.

Comments

The opinions expressed here are the views of the writer and do not necessarily reflect the views and opinions of News Medical.
Post a new comment
Post
You might also like...
Understanding the Choice Between Individual Versus Social Housing Mice and Its Impact On Research