Review of Philosophy and Psychology 12 (1):101-120 (2021)

Deborah Mayo
Virginia Tech
The crisis of replication has led many to blame statistical significance tests for making it too easy to find impressive looking effects that do not replicate. However, the very fact it becomes difficult to replicate effects when features of the tests are tied down actually serves to vindicate statistical significance tests. While statistical significance tests, used correctly, serve to bound the probabilities of erroneous interpretations of data, this error control is nullified by data-dredging, multiple testing, and other biasing selection effects. Arguments claiming to vitiate statistical significance tests attack straw person variants of tests that commit well-known fallacies and misinterpretations. There is a tension between popular calls for preregistration – arguably, one of the most promising ways to boost replication – and accounts that downplay error probabilities: Bayes Factors, Bayesian posteriors, likelihood ratios. By underscoring the importance of error control for well testedness, the replication crisis points to reformulating tests so as to avoid fallacies and report the extent of discrepancies that are and are not indicated with severity.
Keywords No keywords specified (fix it)
Categories (categorize this paper)
DOI 10.1007/s13164-020-00501-w
Edit this record
Mark as duplicate
Export citation
Find it on Scholar
Request removal from index
Revision history

Download options

PhilArchive copy

Upload a copy of this paper     Check publisher's policy     Papers currently archived: 59,848
External links

Setup an account with your affiliations in order to access resources via your University's proxy server
Configure custom proxy (use this if your affiliation does not provide a proxy)
Through your library

References found in this work BETA

Error and the Growth of Experimental Knowledge.Deborah Mayo - 1996 - International Studies in the Philosophy of Science 15 (1):455-459.
Severe Testing as a Basic Concept in a Neyman–Pearson Philosophy of Induction.Deborah G. Mayo & Aris Spanos - 2006 - British Journal for the Philosophy of Science 57 (2):323-357.

Add more references

Citations of this work BETA

No citations found.

Add more citations

Similar books and articles

Why Replication is Overrated.Uljana Feest - 2019 - Philosophy of Science 86 (5):895-905.
An a Priori Solution to the Replication Crisis.David Trafimow - 2018 - Philosophical Psychology 31 (8):1188-1214.
Replicability and Replication in the Humanities.Rik Peels - 2019 - Research Integrity and Peer Review 4 (1).
The Replication Crisis: How Might Philosophy and Theory of Psychology Be of Use?Jill Morawski - 2019 - Journal of Theoretical and Philosophical Psychology 39 (4):218-238.
Introduction to the Special Section, “Psychology’s Replication Crisis”.Joshua W. Clegg & Kathleen L. Slaney - 2019 - Journal of Theoretical and Philosophical Psychology 39 (4):199-201.
Significance Tests: Necessary but Not Sufficient.Louis G. Tassinary - 1998 - Behavioral and Brain Sciences 21 (2):221-222.
Statistical Inference and the Replication Crisis.Lincoln J. Colling & Dénes Szűcs - 2021 - Review of Philosophy and Psychology 12 (1):121-147.
Bayesian Tests to Quantify the Result of a Replication Attempt.Josine Verhagen & Eric-Jan Wagenmakers - 2014 - Journal of Experimental Psychology: General 143 (4):1457-1475.


Added to PP index

Total views
13 ( #729,279 of 2,433,036 )

Recent downloads (6 months)
5 ( #138,245 of 2,433,036 )

How can I increase my downloads?


My notes