Psychology has a replication problem. Since 2010, scientists conducting replications of hundreds of studies have discovered that a dismal amount of published results can be reproduced. This realization by psychologists has come to be known as "replication crisis". For me, this story all started with ego-depletion, and the comics I had drawn about it in 2014. The idea is that your self-control is a resource that can be diminished with use. When you think about all the times you've been slowly worn down by temptation, it seems obvious. When I drew the comics, there had been new research pointing to blood sugar levels as the font of self-control from which we all drew from. It also made sense—people get cranky when they're hungry. We even made up a word for it. We call it being "hangry".
Early in my Ph.D. studies, my supervisor assigned me the task of running computer code written by a previous student who was graduated and gone. It was hell. I had to sort through many different versions of the code, saved in folders with a mysterious numbering scheme. There was no documentation and scarcely an explanatory comment in the code itself. It took me at least a year to run the code reliably, and more to get results that reproduced those in my predecessor's thesis. Now that I run my own lab, I make sure that my students don't have to go through that.
A comic illustrating the complexities and history of research reproducibility.
Scientists, public servants, and patient advocates alike increasingly question the validity of published scientific results, endangering the public’s acceptance of science. Here, I argue that emerging flaws in the integrity of the peer review system are largely responsible. Distortions in peer review are driven by economic forces and enabled by a lack of accountability of journals, editors, and authors. One approach to restoring trust in the validity of published results may be to establish basic rules that render peer review more transparent, such as publishing the reviews (a practice already embraced by some journals) and monitoring not only the track records of authors but also of editors and journals.
Sanjay Srivastava’s joke syllabus ("A Joke Syllabus With a Serious Point: Cussing Away the Reproducibility Crisis," The Chronicle, August 15) and Lee Jussim's blog post on Psychology Today about educating psychology students in light of the reproducibility crisis led me to reflect on my department’s recent curriculum changes. We have retooled or created from scratch multiple courses that engage something few of my colleagues seem to consider relevant to the problem: intellectual history. They instead hold firmly to the dictates of positivism, insisting that better training in the methods of science will be the source of rescue. Where does this prejudice come from? Might it be time to pave a new way?
We heard back from 270 scientists around the world, including graduate students, senior professors, laboratory heads, and Fields Medalists. And they told us that in a variety of ways, they feel their careers are being hijacked by perverse incentives.