Research is the driving force in academia with professors and researchers invested in a certain area of expertise.
Although the methods of research differ for different fields, the process is generally the same with the researcher conducting studies, submitting findings to peer review, and then publishing their findings in an academic journal related to the area of the study.
And for most of the time this system has been in place, it has worked as designed. That is, up until recently.
Research has entered a crisis period when it comes to the ability to reproduce the results of a given study. Science has always moved forward when scientists corroborate what other scientists find.
It is not possible to take the results of one single study and proclaim them to be a 100 percent certain.
For example, if I peek into a sealed box and say that there is a blue ball inside, that does not mean that you can say the box contains a blue ball. The next observer could come by to the same box and say there is a yellow cube. But if 100 people say that the sealed box contains a blue ball then my initial observation has been successfully replicated and it is more than likely true.
I had the chance to speak to Dr. Matt McBee, a member of the ETSU Psychology department faculty, to get his take on the reproducibility crisis, specifically in the field of psychology.
According to the Atlantic, the replication crisis is mostly caused by the academic journal industry, which incentivizes positive findings that other scientists will cite in their own studies.
“This, of course, creates a huge incentive for researchers to obtain statistical significance and ‘clean up’ their articles so that a nice, clean story is described,” McBee said.
“Finding something makes the researcher happy — they get to publish,” he said. “And it makes the journal happy — they have something to publish. And if the story is exciting, interesting or unexpected, so much the better — it will probably get a lot of citations and help boost the journal’s impact factor. But very often, it seems, the something that is found? It’s nothing. And if someone tries to replicate that result, they won’t be able to. Because it wasn’t really there. As Andrew Gelman put it, it was ‘a tale spun out of noise.’”
According to Nature, a little more than 70 percent of researchers that were surveyed revealed that they were unable to reproduce the results of a study undertaken by another scientist in their field.
“It is a massive problem,” McBee said. “I believe that many more fields will be affected. All fields that rely on the poor incentive structure and ineffective organization that I have described above are going to have a replication crisis. It’s just that many fields outside of some areas of psychology and the life sciences haven’t started critically assessing the state of their literature yet. When they do — they’ll find the same thing.”
So what changes can the scientific community make and what is the outlook for what we know?
“I have created a new lab called the Laboratory for Meta-Science and Reproducible Research,” McBee said. “This lab is devoted to assessing the state of the scientific literature in psychology and education and to developing and testing reforms to improve the reliability and reproducibility of social science. Our lab will be, among other projects, participating in an upcoming multilab replication project. Also, as a professor responsible for teaching undergraduate and graduate courses on statistics and research methods — my teaching has changed as well. The replication crisis (and its antidote: open science and reproducible research) is at the core of everything I teach.”
McBee thinks that by working towards fixing the problems that caused the replication crisis, we can have a better future and relationship with scientific findings.
“It is a problem that can be solved,” he said. “We can (and are!) creating better systems for doing science.”
