Review of General Psychology 21:269-275 (2017)

Authors
Mark Rubin
University of Newcastle, Australia
Abstract
Several researchers have recently argued that p values lose their meaning in exploratory analyses due to an unknown inflation of the alpha level (e.g., Nosek & Lakens, 2014; Wagenmakers, 2016). For this argument to be tenable, the familywise error rate must be defined in relation to the number of hypotheses that are tested in the same study or article. Under this conceptualization, the familywise error rate is usually unknowable in exploratory analyses because it is usually unclear how many hypotheses have been tested on a spontaneous basis and then omitted from the final research report. In the present article, I argue that it is inappropriate to conceptualize the familywise error rate in relation to the number of hypotheses that are tested. Instead, it is more appropriate to conceptualize familywise error in relation to the number of different tests that are conducted on the same null hypothesis in the same study. Under this conceptualization, alpha level adjustments in exploratory analyses are (a) less necessary and (b) objectively verifiable. As a result, p values do not lose their meaning in exploratory analyses.
Keywords No keywords specified (fix it)
Categories (categorize this paper)
Options
Edit this record
Mark as duplicate
Export citation
Find it on Scholar
Request removal from index
Revision history

Download options

PhilArchive copy

 PhilArchive page | Other versions
External links

Setup an account with your affiliations in order to access resources via your University's proxy server
Configure custom proxy (use this if your affiliation does not provide a proxy)
Through your library

References found in this work BETA

No references found.

Add more references

Citations of this work BETA

The Costs of HARKing.Mark Rubin - forthcoming - British Journal for the Philosophy of Science:000-000.

View all 8 citations / Add more citations

Similar books and articles

In Defence of Error Theory.Chris Daly & David Liggins - 2010 - Philosophical Studies 149 (2):209-230.
A Rate of Incoherence Applied to Fixed-Level Testing.Mark J. Schervish, Teddy Seidenfeld & Joseph B. Kadane - 2002 - Proceedings of the Philosophy of Science Association 2002 (3):S248-S264.
Inductive Risk and Values in Science.Heather Douglas - 2000 - Philosophy of Science 67 (4):559-579.

Analytics

Added to PP index
2019-12-10

Total views
280 ( #32,933 of 2,445,529 )

Recent downloads (6 months)
83 ( #7,648 of 2,445,529 )

How can I increase my downloads?

Downloads

My notes