Is There a Free Lunch in Inference?

Topics in Cognitive Science 8 (3):520-547 (2016)
  Copy   BIBTEX

Abstract

The field of psychology, including cognitive science, is vexed by a crisis of confidence. Although the causes and solutions are varied, we focus here on a common logical problem in inference. The default mode of inference is significance testing, which has a free lunch property where researchers need not make detailed assumptions about the alternative to test the null hypothesis. We present the argument that there is no free lunch; that is, valid testing requires that researchers test the null against a well-specified alternative. We show how this requirement follows from the basic tenets of conventional and Bayesian probability. Moreover, we show in both the conventional and Bayesian framework that not specifying the alternative may lead to rejections of the null hypothesis with scant evidence. We review both frequentist and Bayesian approaches to specifying alternatives, and we show how such specifications improve inference. The field of cognitive science will benefit because consideration of reasonable alternatives will undoubtedly sharpen the intellectual underpinnings of research.

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 90,221

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

When can non‐commutative statistical inference be Bayesian?Miklós Rédei - 1992 - International Studies in the Philosophy of Science 6 (2):129-132.
Reverse Inference in Neuropsychology.Clark Glymour & Catherine Hanson - 2016 - British Journal for the Philosophy of Science 67 (4):1139-1153.
Inference to the Best Explanation.Peter Lipton - 1991 - London and New York: Routledge/Taylor and Francis Group.
In Defense of Reverse Inference.Edouard Machery - 2014 - British Journal for the Philosophy of Science 65 (2):251-267.

Analytics

Added to PP
2016-08-05

Downloads
37 (#372,796)

6 months
3 (#439,386)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

Richard Morey
Cardiff University