Type I error rates are not usually inflated

Abstract

The inflation of Type I error rates is thought to be one of the causes of the replication crisis. Questionable research practices such as p-hacking are thought to inflate Type I error rates above their nominal level, leading to unexpectedly high levels of false positives in the literature and, consequently, unexpectedly low replication rates. In this article, I offer an alternative view. I argue that questionable and other research practices do not usually inflate relevant Type I error rates. I begin with an introduction to Type I error rates that distinguishes them from theoretical errors. I then illustrate my argument with respect to model misspecification, multiple testing, selective inference, forking paths, exploratory analyses, p-hacking, optional stopping, double dipping, and HARKing. In each case, I demonstrate that relevant Type I error rates are not usually inflated above their nominal level, and in the rare cases that they are, the inflation is easily identified and resolved. I conclude that the replication crisis may be explained, at least in part, by researchers’ misinterpretation of statistical errors and their underestimation of theoretical errors.

Links

PhilArchive

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

  • Only published works are available at libraries.

Similar books and articles

Error Rates and Uncertainty Reduction in Rule Discovery.Emrah Aktunc - forthcoming - Review of Philosophy and Psychology.
Error Rates and Uncertainty Reduction in Rule Discovery.M. Emrah Aktunc, Ceren Hazar & Emre Baytimur - 2020 - Review of Philosophy and Psychology 12 (2):435-452.
The Naturalizing Error.Douglas Allchin & Alexander J. Werth - 2017 - Journal for General Philosophy of Science / Zeitschrift für Allgemeine Wissenschaftstheorie 48 (1):3-18.
Island Biogeography, Species-Area Curves, and Statistical Errors: Applied Biology and Scientific Rationality.Kristin S. Shrader-Frechette - 1990 - PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association 1990:447 - 456.
The Error In 'The Error In The Error Theory'.Richard Joyce - 2011 - Australasian Journal of Philosophy 89 (3):519-534.

Analytics

Added to PP
2023-11-21

Downloads
727 (#23,162)

6 months
654 (#1,995)

Historical graph of downloads
How can I increase my downloads?