In his show Last Week Tonight, John Oliver discusses how and why media outlets so often report untrue or incomplete information as science.
Reproducible Science Promoting Open Science
The aim of this study was to assess the reproducibility and validity of a non-quantitative 28-item food frequency questionnaire (FFQ). Children aged 9–10 years (n = 50) from three schools in Dunedin, New Zealand, completed the FFQ twice and a four-day estimated food diary (4DEFD) over a two-week period. Intraclass correlation coefficients (ICC) and Spearman’s correlation coefficients (SCC) were used to determine reproducibility and validity of the FFQ, respectively.
Kyle Cranmer, a faculty member in NYU's physics department, distills and describes of the event of NYU's first Reproducibility Symposium on May 3, 2016.
This Workshop aims at becoming a forum to discuss ideas and advancements towards the revision of current scientific communication practices in order to support Open Science, introduce novel evaluation schemes, and enable reproducibility. As such it candidates as an event fostering collaboration between (i) Library and information scientists working on the identification of new publication paradigms; (ii) ICT scientists involved in the definition of new technical solutions to these issues; (iii) scientists/researchers who actually conduct the research and demand tools and practices for Open Science. The expected results are advancements in the definition of the next generation scientific communication ecosystem, where scientists can publish research results (including the scientific article, the data, the methods, and any “alternative” product that may be relevant to the conducted research) in order to enable reproducibility (effective reuse and decrease of cost of science) and rely on novel scientific reward practices.
Only mandatory Open Data, not Gold Open Access, will lead to more honest and more reproducible science. Open Science is these days largely about mandatory publishing in Open Access (OA), regardless of the costs to poorer scientists or the universities which already struggle to pay horrendous subscription fees. Meanwhile, publishers openly declare that the so-called Gold (author-pays) OA will be much more expensive than even current subscription rates, yet wealthy western institutions like the Dutch university network VSNU or the German Max Planck Society do not seem troubled by this at all. They seriously expect the publishing oligopoly of Elsevier, SpringerNature and Wiley to lower the costs for Gold OA later on, out of the goodness of their hearts (as this winter’s invitation-only Berlin12 OA conference suggests).
Recent reports in the Washington Post and the Economist, among others, raise the concern that relatively few scientists' experimental findings can be replicated. This is worrying: replicating an experiment is a main foundation of the scientific method. As scientists, we build on knowledge gained and published by others. We develop new experiments and questions based on the knowledge we gain from those published reports. If those papers are valid, our work is supported and knowledge advances. On the other hand, if published research is not actually valid, if it can’t be replicated, it delivers only an incidental finding, not scientific knowledge.
There is growing interest in research transparency and reproducibility in economics and other scientific fields. We survey existing work on these topics within economics and discuss the evidence suggesting that publication bias, inability to replicate, and specification searching remain widespread problems in the discipline. We next discuss recent progress in this area, including improved research design, study registration and pre-analysis plans, disclosure standards, and open sharing of data and materials, and draw on experiences in both economics and other social sciences. We discuss areas where consensus is emerging on new practices as well as approaches that remain controversial and speculate about the most effective ways to make economics research more accurate, credible, and reproducible in the future.
There’s a replication crisis in biomedicine—and no one even knows how deep it runs. Many science funders share Parker’s antsiness over all the waste of time and money. In February, the White House announced its plan to put $1 billion toward a similar objective—a “Cancer Moonshot” aimed at making research more techy and efficient. But recent studies of the research enterprise reveal a more confounding issue, and one that won’t be solved with bigger grants and increasingly disruptive attitudes. The deeper problem is that much of cancer research in the lab—maybe even most of it—simply can’t be trusted. The data are corrupt. The findings are unstable. The science doesn’t work.
Recent years have seen an increase in alarming signals regarding the lack of replicability in neuroscience, psychology, and other related fields. To avoid a widespread crisis in neuroimaging research and consequent loss of credibility in the public eye, we need to improve how we do science. This article aims to be a practical guide for researchers at any stage of their careers that will help them make their research more reproducible and transparent while minimizing the additional effort that this might require. The guide covers three major topics in open science (data, code, and publications) and offers practical advice as well as highlighting advantages of adopting more open research practices that go beyond improved transparency and reproducibility.
Traditionally, checkmate is a position in the game of chess in which a player’s king is in check, without a way to remove the threat. The king cannot be captured, so the game ends when the king is checkmated. As a premium surgeon, no one ever wants to be checkmated at any stage of the surgical process, from preoperative to intraoperative to postoperative. Other forms of etymology have suggested checkmate to signify being “ambushed,” a feeling many of us have experienced in our surgical careers. A means to avoiding being a checkmated surgeon is creating “checklists” from the time of the first patient encounter until the final postoperative visit. The process of checklists can bring reproducibility to a surgical process that already yields successful outcomes in a premium surgeon’s practice.