Trust But Verify: The Crisis in Biomedicine

Fool me once, shame on me. Fool me twice, shame on scientific research? Research studies investigating other research studies – yes, it’s very meta – have diagnosed a “reproducibility” crisis in biomedicine. Simply put, it’s shockingly difficult to reproduce the results of many laboratory research studies relied upon as authoritative and definitive.

As a result, the reliability of a large share of the research published about medicine and the biology of medicine is in question. The reasons vary from poor training of young scientists engaged in ever more complex and esoteric inquiries to perverse incentives that reward researchers for flashy findings but don’t penalize them for being wrong. Some errors are inevitable—in fact, it’s a part of science. But today’s reproducibility crisis is challenging the very idea that scientific knowledge expands by research studies that build upon each other. Is a transformation of the underlying culture of biomedical research necessary? How difficult will it be to accomplish?

Opening remarks provided by Richard Harris, Science Correspondent, National Public Radio (on leave); Visiting Scholar, Consortium for Science, Policy & Outcomes, Arizona State University.

Panel One - Verify: How Should the Scientific Process Change to Address the Reproducibility Crisis? Panelists included Emma Frow, Assistant Professor, School of Biological and Health Systems Engineering and School for the Future of Innovation in Society, Arizona State University; Brian Nosek, Professor, Department of Psychology, University of Virginia; Executive Director and Co-founder, Center for Open Science; and, Lawrence A. Tabak, Principal Deputy Director, National Institutes of Health. Moderated by Carolyn Johnson, Business of health reporter, The Washington Post

Panel Two - Trust: How Science Journalism Can Help the Public Evaluate Findings. Panelists included Arturo Casadevall, Alfred & Jill Sommer Professor and Chair, W. Harry Feinstone Department of Microbiology and Immunology, Johns Hopkins University; Bloomberg Distinguished Professor, Department of Medicine, Johns Hopkins University; Carolyn Compton, Professor, Life Science, Arizona State University; Professor, Laboratory Medicine and Pathology, Mayo Clinic School of Medicine; Chief Medical Officer, Complex Adaptive Systems Institute, Arizona State University; and,  Richard Harris, Science Correspondent, National Public Radio (on leave), Visiting Scholar, Consortium for Science, Policy & Outcomes, Arizona State University. Moderated by Rachel Gross, Editorial Assistant, Slate; Board member, DC Science Writers Association.