Credibility Crisis in Science with Thomas Plümper and Eric Neumayer

In this episode of Breaking Math, hosts Autumn Finaf and Noah John-Syracuse explore the credibility crisis in science with guests Thomas Plümper and Eric Neumayer, authors of a book exposing the hidden prevalence of subtle scientific misconduct. They argue that while high-profile fraud cases like Diederik Stapel’s data fabrication grab headlines, the more insidious and widespread issue is 'tweaking'—small, often unconscious changes to model specifications to achieve statistically significant results. The hosts and guests dissect the flaws in the current scientific system, particularly the overreliance on p-values and the illusion of self-correction, showing how even well-intentioned researchers can distort findings through methodological flexibility. The discussion reveals that detection rates are skewed by research type—experimental fields like psychology are more likely to be caught, not because they're more fraudulent, but because their methods are easier to scrutinize. The episode challenges the black-and-white thinking of 'true vs. false' in science, advocating instead for Bayesian approaches that embrace uncertainty and humility. Practical solutions include radical data transparency, independent robustness testing, and a cultural shift toward recognizing tweaking as serious misconduct. Key takeaways include: 1) Statistical significance is often a misleading proxy for truth and can be easily manipulated through model tweaking; 2) The scientific community must treat subtle methodological adjustments as seriously as outright fraud; 3) Robustness testing should be mandated by journals, not self-selected by authors; 4) Data transparency and public archiving are essential to prevent hidden manipulation; 5) The scientific process is not self-correcting fast enough to prevent real-world harm, as seen in the lingering impact of discredited studies like the vaccine-autism link; 6) AI may level the playing field between fraudsters and detectors, but the arms race continues; 7) Humility and probabilistic thinking—central to Bayesian philosophy—are vital for credible science; 8) Fixing the system requires systemic change, not just individual honesty, since human nature won’t change overnight.