A movement for open science and reproducibility has been growing across fields of research. Reproducibility and replicability are part of the ways in which science self-corrects but are not the sole concern. Professor Lorena Barba was a member of the National Academies of Sciences, Engineering and Medicine (NASEM) Reproducibility and Replicability in Science study committee. This committee conducted an in-depth study looking across all of science to better understand what, if any, improvements are needed.
The committee held a series of briefings leading up to the release of their report in 2019 with the White House Office of Science and Technology Policy, the National Science Foundation (NSF) and other stakeholders. It aimed to identify any issues of replication and reproducibility in scientific and engineering research. The report made recommendations for improving rigor and transparency in scientific and engineering research, and identified and highlighted good practices.
The study was commissioned by the National Science Foundation, in response to congressional mandate. Public Law 114-329 cites “growing concern that some published research findings cannot be reproduced or replicated…” and directs the NSF to produce a report with an assessment and recommendations on the matter.
The committee’s study found that improvements are needed—more transparency of computational workflows, code and data, for example, and adjusting the incentive structure to value reproducible research. Other findings include the need for greater fluency with statistics, and training of early career researchers on computational tools and methods. The study has been influential in policy changes and new programs, including the new NASA Transform to Open Science (TOPS) mission.