Researchers tease out different definitions of a crucial scientific term. A semantic confusion is clouding one of the most talked-about issues in research. Scientists agree that there is a crisis in reproducibility, but they can’t agree on what 'reproducibility' means.
A huge audience of psychologists, students and researchers was drawn to the British Psychological Society debate in London about the reproducibility and replication crisis in psychology. After Brian Nosek and the Open Science Collaboration outlined the difficulty in reproducing psychological findings, the BPS, the Experimental Psychology Society and the Association of Heads of Psychology Departments hoped to host an upbeat and positive debate in the area. Ella Rhodes reports from a British Psychological Society debate.
Here is the past week’s career-related news from across the Science family of publications.
Two new research papers on scabies and tapeworms published today showcase a new collaboration with protocols.io. This demonstrates a new way to share scientific methods that allows scientists to better repeat and build upon these complicated studies on difficult-to-study parasites. It also highlights a new means of writing all research papers with citable methods that can be updated over time.
A few years ago, the topic of whether scientific papers are reproducible or not would have been an odd thing to see in a newspaper. But not any more: both the popular media and the journals themselves have been trying to deal with the topic, amid reports that far too many results can’t be replicated. Large scale efforts have begun to examine key papers in experimental psychology, among other areas. Reports from the biopharma industry about the numbers of interesting biology papers that don’t hold up have stirred alarm as well. But as far as I can tell, chemistry has largely escaped the current rounds of criticism.
Here are the results of a Nature survey on reproducibility in the scientific literature. They themselves admit that it’s a "confusing snapshot", but it shows that we're still arguing about what "reproducibility" means. 52% of the responders (over 1500 scientists) said that there was "a significant crisis", though, so this issue is on people’s minds. Interestingly, chemists were among the most confidant in the literature of their own field (physics and engineering as well). At the same time, chemists had the highest proportion of respondents who said that they'd been unable to reproduce someone else's experiment. I don't think that's necessarily a contradiction, though. Chemistry is a field with lower barriers to replication than many others, and we also probably do more replications in general.