Why so many studies can’t be replicated

This episode of Science Friday explores the growing replication crisis in scientific research, focusing on the findings of the large-scale SCORE Project funded by DARPA. The project analyzed thousands of studies across economics, education, and psychology, revealing that only about half could be successfully replicated. Host Ira Flatow discusses the implications with Dr. Tim Arrington of the Center for Open Science and Dr. Abel Brodeur of the Institute for Replication, who highlight systemic issues such as poor data and methodological transparency, lack of code sharing, and the pressure to publish positive results. Despite these challenges, both researchers note improvements in recent years, particularly in data sharing and code review, driven by evolving norms and institutional incentives. The conversation shifts to the role of AI, which presents both threats—like generating misleading research language—and opportunities, such as automating replication checks and exploring multiple analytical approaches to test robustness. The episode concludes with a call for patience and skepticism in interpreting scientific headlines, emphasizing that trust in science grows through repeated validation rather than single studies. Key takeaways include: (1) Replication is essential for scientific credibility, and most studies cannot be replicated due to poor transparency; (2) Data and code sharing are critical but still inconsistent across disciplines; (3) The scientific community is gradually improving through better norms, training, and institutional support; (4) AI can both threaten and enhance replication efforts; and (5) The public should approach scientific claims with healthy skepticism and wait for replication before placing trust. The overall tone is constructive and hopeful, acknowledging flaws while highlighting meaningful progress and solutions.