It is generally accepted among philosophers of science that hypothesis testing is a key methodological feature of science. As far as philosophical theories of confirmation are con...

Preregistration has been proposed as a useful method for making a publicly verifiable distinction between confirmatory hypothesis tests, which involve planned tests of ante hoc hypotheses, and exploratory hypothesis tests, which involve unplanned tests of post hoc hypotheses. This distinction is thought to be important because it has been proposed that confirmatory hypothesis tests provide more compelling results (less uncertain, less tentative, less open to bias) than exploratory hypothesis tests. In this article, we challenge this proposition (...) and argue that there are several advantages of exploratory hypothesis tests that can make their results more compelling than those of confirmatory hypothesis tests. We also consider some potential disadvantages of exploratory hypothesis tests and conclude that their advantages can outweigh the disadvantages. We conclude that exploratory hypothesis tests avoid researcher commitment and researcher prophecy biases, reduce the probability of data fraud, are more appropriate in the context of unplanned deviations, facilitate inference to the best explanation, and allow peer reviewers to make additional contributions at the data analysis stage. In contrast, confirmatory hypothesis tests may lead to an inappropriate level of confidence in research conclusions, less appropriate analyses in the context of unplanned deviations, and greater bias and errors in theoretical inferences. (shrink)

According to one theory, the brain is a sophisticated hypothesis tester: perception is Bayesian unconscious inference where the brain actively uses predictions to test, and then refine, models about what the causes of its sensory input might be. The brain’s task is simply continually to minimise prediction error. This theory, which is getting increasingly popular, holds great explanatory promise for a number of central areas of research at the intersection of philosophy and cognitive neuroscience. I show how the theory (...) can help us understand striking phenomena at three cognitive levels: vision, sensory integration, and belief. First, I illustrate central aspects of the theory by showing how it provides a nice explanation of why binocular rivalry occurs. Then I suggest how the theory may explain the role of the unified sense of self in rubber hand and full body illusions driven by visuotactile conflict. Finally, I show how it provides an approach to delusion formation that is consistent with one-deficit accounts of monothematic delusions. (shrink)

“Dutch Book” arguments and references to gambling theorems are typical in the debate between Bayesians and scientists committed to “classical” statistical methods. These arguments have rarely convinced non-Bayesian scientists to abandon certain conventional practices, partially because many scientists feel that gambling theorems have little relevance to their research activities. In other words, scientists “don’t bet.” This article examines one attempt, by Schervish, Seidenfeld, and Kadane, to progress beyond such apparent stalemates by connecting “Dutch Book”–type mathematical results with principles actually endorsed (...) by practicing experimentalists. (shrink)

Unfortunately, reading Chow's work is likely to leave the reader more confused than enlightened. My preferred solutions to the “controversy” about null- hypothesis testing are: (1) recognize that we really want to test the hypothesis that an effect is “small,” not null, and (2) use Bayesian methods, which are much more in keeping with the way humans naturally think than are classical statistical methods.

Although many researchers may perceive empirical hypothesis testing using inferential statistics to be a value free process, I argue that any conclusion based on inferential statistics contains an important and intractable value judgment. Consequently, I conclude that researchers should use the same rationale for examining the ethical ramifications of committing errors in statistical inference that they use to examine the ethical parameters of a proposed research design.

Scholars employing ossified ‘close reading’ methods generate countless articles that drop down into a gravity vortex, circling themselves for a self-referential eternity. After arguing that the study of texts in the humanities, especially literature and philosophy, makes no progress, I set this controversy in the light of a distinction between the Soft and Hard Humanities. This is not an a priori argument from an ivory tower. Rather than tell, I show. I present data from the testing of a (...) class='Hi'>hypothesis drawn from genre theory about science fi ction, then say a few words about the uncertain future of philosophy and literature. (shrink)

Humans appear to follow normative rules of inductive reasoning in "premise diversity tasks" that is, they know that dissimilar rather than similar evidence is better for generalising hypotheses. In three experiments, we use a "hypothesis limitation task" to compare a related inductive reasoning skill knowing how to limit hypotheses by using a negative test strategy. Participants are told that one category member has some property (e.g. Dogs have a merocrine gland) and are asked what evidence they would test to (...) ensure that either all (generalisation) or only (limitation) category members have that property (e.g. All/Only mammals have merocrine glands; tests: wolf, bull, crocodile). Despite participants' reluctance to use negative tests in the Wason 2-4-6 task and other reasoning tasks, participants do use normatively correct negative tests in the hypothesis limitation task as often as they use diverse positive tests in the premise diversity task. Moreover, when given a hypothesis limitation task before a rule evaluation task (similar to the 2-4-6 task), the use of negative tests increases. Thus, when testing hypotheses, people can and do use the right kind of test strategy for the task. (shrink)

Chow's defense of NHSTP ignores the fact that in psychology it is used to test substantive hypotheses in theory-corroborating research. In this role, NHSTP is not only inadequate, but damaging to the progress of psychology as a science. NHSTP does not fulfill the Popperian requirement that theories be tested severely. It also encourages nonspecific predictions and feeble theoretical formulations.

Although logical consistency is desirable in scientific research, standard statistical hypothesis tests are typically logically inconsistent. To address this issue, previous work introduced agnostic hypothesis tests and proved that they can be logically consistent while retaining statistical optimality properties. This article characterizes the credal modalities in agnostic hypothesis tests and uses the hexagon of oppositions to explain the logical relations between these modalities. Geometric solids that are composed of hexagons of oppositions illustrate the conditions for these modalities (...) to be logically consistent. Prisms composed of hexagons of oppositions show how the credal modalities obtained from two agnostic tests vary according to their threshold values. Nested hexagons of oppositions summarize logical relations between the credal modalities in these tests and prove new relations. (shrink)

Complex constraints like conditionals ('If A, then B') and probabilistic constraints ('The probability that A is p') pose problems for Bayesian theories of learning. Since these propositions do not express constraints on outcomes, agents cannot simply conditionalize on the new information. Furthermore, a natural extension of conditionalization, relative information minimization, leads to many counterintuitive predictions, evidenced by the sundowners problem and the Judy Benjamin problem. Building on the notion of a `paradigm shift' and empirical research in psychology and economics, I (...) argue that the model of hypothesis testing can explain how people learn complex, theory-laden propositions like conditionals and probability constraints. Theories are formalized as probability distributions over a set of possible outcomes and theory change is triggered by a constraint which is incompatible with the initial theory. This leads agents to consult a higher order probability function, or a 'prior over priors,' to choose the most likely alternative theory which satisfies the constraint. The hypothesis testing model is applied to three examples: learning a simple probabilistic constraint involving coin bias, the sundowners problem for conditional learning, and the Judy Benjamin problem for learning conditional probability constraints. (shrink)

Koriat & Goldsmith's distinction between the correspondence and storehouse metaphors is valuable for both memory theory and methodology. It is questionable, however, whether this distinction underlies the heated debate about so called “everyday memory” research. The distinction between experimental and naturalistic methodologies better characterizes this debate. I compare these distinctions and discuss how the methodological distinction, between experimental and naturalistic designs, could give rise to different theoretical approaches.

According to the pragmatic hypothesis testing theory, how much evidence we require before we believe something varies depending on the expected costs of falsely believing and disbelieving it. This theory has been used in the self-deception debate to explain our tendencies towards self-deceptive belief formation. This article argues that the application of this theory in the self-deception debate has overlooked the distinction between belief and acceptance, and that the theory in all likelihood models acceptance rather than belief, in (...) which case it is probably not relevant to the explanation of self-deception. It is suggested, however, that doxastic error costs might be relevant to explaining some types of self-deception, though they feature in an evolutionary explanation of it rather than a psychological one. (shrink)

We show that if among the tested hypotheses the number of true hypotheses is not equal to the number of false hypotheses, then Neyman-Pearson theory of testing hypotheses does not warrant minimal epistemic reliability. We also argue that N-P does not protect from the possible negative effects of the pragmatic value-laden unequal setting of error probabilities on N-P’s epistemic reliability. Most importantly, we argue that in the case of a negative impact no methodological adjustment is available to neutralize it, (...) so in such cases the discussed pragmatic-value-ladenness of N-P inevitably compromises the goal of attaining truth. (shrink)

Three correctives can get researchers out of the trap of constructing unitary theories of “thinking”: (1) Strong inference methods largely avoid problems associated with universal prescriptive normativism; (2) theories must recognize that significant modularity of cognitive processes is antithetical to general accounts of thinking; and (3) consideration of the domain-specificity of rationality render many of the present article's issues moot.

In the past, hypothesis testing in medicine has employed the paradigm of the repeatable experiment. In statistical hypothesis testing, an unbiased sample is drawn from a larger source population, and a calculated statistic is compared to a preassigned critical region, on the assumption that the comparison could be repeated an indefinite number of times. However, repeated experiments often cannot be performed on human beings, due to ethical or economic constraints. We describe a new paradigm for (...) class='Hi'>hypothesis testing which uses only rearrangements of data present within the observed data set. The token swap test, based on this new paradigm, is applied to three data sets from cardiovascular pathology, and computational experiments suggest that the token swap test satisfies the Neyman Pearson condition. (shrink)

Simultaneous hypothesis tests can fail to provide results that meet logical requirements. For example, if A and B are two statements such that A implies B, there exist tests that, based on the same data, reject B but not A. Such outcomes are generally inconvenient to statisticians (who want to communicate the results to practitioners in a simple fashion) and non-statisticians (confused by conflicting pieces of information). Based on this inconvenience, one might want to use tests that satisfy logical (...) requirements. However, Izbicki and Esteves shows that the only tests that are in accordance with three logical requirements (monotonicity, invertibility and consonance) are trivial tests based on point estimation, which generally lack statistical optimality. As a possible solution to this dilemma, this paper adapts the above logical requirements to agnostic tests, in which one can accept, reject or remain agnostic with respect to a given hypothesis. Each of the logical requirements is characterized in terms of a Bayesian decision theoretic perspective. Contrary to the results obtained for regular hypothesis tests, there exist agnostic tests that satisfy all logical requirements and also perform well statistically. In particular, agnostic tests that fulfill all logical requirements are characterized as region estimator-based tests. Examples of such tests are provided. (shrink)

This paper addresses the role of specification tests in the selection of a statistically admissible model used to evaluate economic hypotheses. The issue is formulated in the context of recent philosophical accounts on the nature of models and related to some results in the literature on specification search. In contrast to enumerative induction and a priori theory, powerful search methodologies are often adequate substitutes for experimental methods. They underwrite and support, rather than distort, statistical hypothesis tests. Their success is (...) grounded in a systematic effort to mold appropriate models to, and test them against, constraints of the data. (shrink)

The confounding effects of heterogeneity in biological psychiatry and psychiatric genetics have been widely discussed in the literature. We suggest an approach in which heterogeneity may be put to use in hypothesis testing, and may find application in evaluation of the Crespi & Badcock (C&B) imprinting hypothesis. Here we consider three potential sources of etiologic subtypes for analysis.

“Sociosexuality from Argentina to Zimbabwe: A 48-nation study of sex, culture, and strategies of human mating” delivers on its title. By combining empiricism and careful hypothesis testing, it not only contributes to our current knowledge but also points the way to further advances.

We introduce an expected-value theory of linguistic modality that makes reference to expected utility and a likelihood-based confirmation measure for deontics and epistemics, respectively. The account is a probabilistic semantics for deontics and epistemics, yet it proposes that deontics and epistemics share a common core modal semantics, as in traditional possible-worlds analysis of modality. We argue that this account is not only theoretically advantageous, but also has far-reaching empirical consequences. In particular, we predict modal versions of reasoning fallacies from the (...) heuristics and biases literature. Additionally, we derive the modal semantics in an entirely transparent manner, as it is based on the compositional semantics of Korean modal expressions that are morphosyntactically decomposed into a conditional and an evaluative predicate. (shrink)

In this commentary, I agree with Chow's treatment of null hypothesis significance testing as a noninferential procedure. However, I dispute his reconstruction of the logic of theory corroboration. I also challenge recent criticisms of NHSTP based on power analysis and meta-analysis.

Professor Meehl [2] has pointed out a very significant problem in the methodology of psychological research, indicating that statistical tests of psychological hypotheses against a null hypothesis are loaded in favor of eventual success at rejecting the null hypothesis. In my opinion this is not, however, a contrast between physics and psychology, but rather between the method of parameter estimation and that of the null hypothesis in the tradition of Fisher. A physicist could use the null (...) class='Hi'>hypothesis method as well as the psychologist. The fact that he doesn't is probably related to the more advanced state of his measurement techniques and theoretical constructs. (shrink)

The present book is a collection of essays exploring some classical dimensions of mind both from the perspective of an empirically-informed philosophy and from the point of view of a philosophically-informed psychology. -/- In the last three decades, the level of interaction between philosophy and psychology has increased dramatically. As a contribution to this trend, this book explores some areas in which this interaction has been very productive – or, at least, highly provocative. -/- The interaction between philosophy and psychology (...) can be of different kinds. For example, psychology can be the subject for philosophy of science. In such a case, the philosopher of science pursues the usual set of issues (explanation, reduction, etc.) within the special case of psychology. Or, philosophy can be the source of proposals for improving psychology. Vice versa, the findings of psychology can be used to criticize philosophical theories and suggest ways to resolve some traditional philosophical questions about the mind, such as the nature of mental representation, perception, emotion, memory, consciousness and free will. -/- The chapters in this book reflect these different forms of interaction in an effort to clarify issues and debates concerning some traditional cognitive capacities. The result is a philosophically and scientifically up-to-date collection of "cartographies of the mind.". (shrink)

The current strategy of searching for an effective treatment for COVID-19 relies mainly on repurposing existing therapies developed to target other diseases. Conflicting results have emerged in regard to the efficacy of several tested compounds but later results were negative. The number of conducted and ongoing trials and the urgent need for a treatment pose the risk that false-positive results will be incorrectly interpreted as evidence for treatments’ efficacy and a ground for drug approval. Our purpose is twofold. First, we (...) show that the number of drug-repurposing trials can explain the false-positive results. Second, we assess the evidence for treatments’ efficacy from the perspective of evidential pluralism and argue that considering mechanistic evidence is particularly needed in cases when the evidence from clinical trials is conflicting or of low quality. Our analysis is an application of the program of Evidence Based Medicine Plus to the drug repurposing trials for COVID. Our study shows that if decision-makers applied EBM+, authorizing the use of ineffective treatments would be less likely. We analyze the example of trials assessing the efficacy of hydroxychloroquine as a treatment for COVID-19 and mechanistic evidence in favor of and against its therapeutic power to draw a lesson for decision-makers and drug agencies on how excessive hypothesis testing can lead to spurious findings and how studying negative mechanistic evidence can be helpful in discriminating genuine from spurious results. (shrink)

Gangestad & Simpson's account of the role of good-gene sexual selection in conditional human mating strategies is reasonably convincing, but could be more so with a little more attention to (1), dropping unnecessary sub hypotheses and especially (2) the inclusion of alternative evolutionary explanations.

Exploratory inquiry has difficulty attracting research funding because funding agencies have little sense of how to detect good science in exploratory contexts. After documenting and explaining the focus on hypothesis testing among a variety of institutions responsible for distinguishing between good and bad science, I analyze the NIH grant review process. I argue that a good explanation for the focus on hypothesis testing—at least at the level of science funding agencies—is the fact that hypothesis-driven research (...) is relatively easy to appraise. I then explore one method by which we might gauge the epistemic merits of different styles of inquiry. (shrink)