We provide self-contained proof of a theorem relating probabilistic coherence of forecasts to their non-domination by rival forecasts with respect to any proper scoring rule. The theorem recapitulates insights achieved by other investigators, and clarifi es the connection of coherence and proper scoring rules to Bregman divergence.

Eric Martin and Daniel N. Osherson present a theory of inductive logic built on model theory. Their aim is to extend the mathematics of Formal Learning Theory to a more general setting and to provide a more accurate image of empirical inquiry. The formal results of their study illuminate aspects of scientific inquiry that are not covered by the commonly applied Bayesian approach.

It is easy to construct pairs of sentences X, Y that lead many people to ascribe higher probability to the conjunction X-and-Y than to the conjuncts X, Y. Whether an error is thereby committed depends on reasoners’ interpretation of the expressions “probability” and “and.” We report two experiments designed to clarify the normative status of typical responses to conjunction problems. © 2004 Cognitive Science Society, Inc. All rights reserved.

Jeffrey (1983) proposed a generalization of conditioning as a means of updating probability distributions when new evidence drives no event to certainty. His rule requires the stability of certain conditional probabilities through time. We tested this assumption (“invariance”) from the psychological point of view. In Experiment 1 participants offered probability estimates for events in Jeffrey’s candlelight example. Two further scenarios were investigated in Experiment 2, one in which invariance seems justified, the other in which it does not. Results were in (...) rough conformity to Jeffrey (1983)’s principle. (shrink)

Most speakers experience unclarity about the application of predicates like tall and red to liminal cases. We formulate alternative psychological hypotheses about the nature of this unclarity, and report experiments that provide a partial test of them. A psychologized version of the ‘vagueness-as-ignorance’ theory is then advanced and defended.

We consider a panel of experts asked to assign probabilities to events, both logically simple and complex. The events evaluated by different experts are based on overlapping sets of variables but may otherwise be distinct. The union of all the judgments will likely be probabilistic incoherent. We address the problem of revising the probability estimates of the panel so as to produce a coherent set that best represents the group’s expertise.

We describe a logic of preference in which modal connectives reflect reasons to desire that a sentence be true. Various conditions on models are introduced and analyzed.

Because the conjunction pandq implies p, the value of a bet on pandq cannot exceed the value of a bet on p at the same stakes. We tested recognition of this principle in a betting paradigm that (a) discouraged misreading p as pandnotq, and (b) encouraged genuinely conjunctive reading of pandq. Frequent violations were nonetheless observed. The findings appear to discredit the idea that most people spontaneously integrate the logic of conjunction into their assessments of chance.

A familiar adage in the philosophy of science is that general hypotheses are better supported by varied evidence than by uniform evidence. Several studies suggest that young children do not respect this principle, and thus suffer from a defect in their inductive methodology. We argue that the diversity principle does not have the normative status that psychologists attribute to it, and should be replaced by a simple rule of probability. We then report experiments designed to detect conformity to the latter (...) rule in children's inductive judgment. The results suggest that young children in both the United States and Taiwan are sensitive to the constraints imposed by the rule on judgments of probability and evidential strength. We conclude with a suggested reinterpretation of the thesis that children's inductive methodology qualifies them as “little scientists.”. (shrink)

Alternative models of idealized scientific inquiry are investigated and compared. Particular attention is devoted to paradigms in which a scientist is required to determine the truth of a given sentence in the structure giving rise to his data.

Stochastic forecasts in complex environments can benefit from combining the estimates of large groups of forecasters (“judges”). But aggregating multiple opinions faces several challenges. First, human judges are notoriously incoherent when their forecasts involve logically complex events. Second, individual judges may have specialized knowledge, so different judges may produce forecasts for different events. Third, the credibility of individual judges might vary, and one would like to pay greater attention to more trustworthy forecasts. These considerations limit the value of simple aggregation (...) methods like linear averaging. In this paper, a new algorithm is proposed for combining probabilistic assessments from a large pool of judges. Two measures of a judge’s likely credibility are introduced and used in the algorithm to determine the judge’s weight in aggregation. The algorithm was tested on a data set of nearly half a million probability estimates of events related to the 2008 U.S. presidential election (∼ 16000 judges). (shrink)

We advance a model of human probability judgment and apply it to the design of an extrapolation algorithm. Such an algorithm examines a person's judgment about the likelihood of various statements and is then able to predict the same person's judgments about new statements. The algorithm is tested against judgments produced by thirty undergraduates asked to assign probabilities to statements about mammals.

We propose a theory that relates perceived evidence to numerical probability judgment. The most successful prior account of this relation is Support Theory, advanced in Tversky and Koehler. Support Theory, however, implies additive probability estimates for binary partitions. In contrast, superadditivity has been documented in Macchi, Osherson, and Krantz, and both sub- and superadditivity appear in the experiments reported here. Nonadditivity suggests asymmetry in the processing of focal and nonfocal hypotheses, even within binary partitions. We extend Support Theory by revising (...) its basic equation to allow such asymmetry, and compare the two equations’ ability to predict numerical assessments of probability from scaled estimates of evidence for and against a given proposition. Both between- and within-subject experimental designs are employed for this purpose. We find that the revised equation is more accurate than the original Support Theory equation. The implications of asymmetric processing on qualitative assessments of chance are also briefly discussed. (shrink)

We advance a theory of inductive reasoning based on similarity, and test it on arguments involving mammal categories. To measure similarity, we quantified the overlap of neural activation in left Brodmann area 19 and the left ventral temporal cortex in response to pictures of different categories; the choice of of these regions is motivated by previous literature. The theory was tested against probability judgments for 40 arguments generated from 9 mammal categories and a common predicate. The results are interpreted in (...) the context of Hume’s thesis relating similarity to inductive inference. (shrink)

A paradigm of scientific discovery is defined within a first-order logical framework. It is shown that within this paradigm there exists a formal scientist that is Turing computable and universal in the sense that it solves every problem that any scientist can solve. It is also shown that universal scientists exist for no regular logics that extend first-order logic and satisfy the Löwenheim-Skolem condition.

A glance at the sky raises my probability of rain to .7. As it happens, the conditional probabilities of each state given rain remain the same, and similarly for their conditional probabilities given no rain. As Jeffrey (1983, Ch. 11) points out, my new distribution P2 is therefore fixed by the law of total probability. For example, P2(RC) = P2(RC | R)P2(R)+P2(RC | ¯.

The presence of nonobservational vocabulary is shown to be necessary for wide application of a conservative principle of theory revision.

A model of idealized scientific inquiry is presented in which scientists are required to infer the nature of the structure that makes true the data they examine. A necessary and sufficient condition is presented for scientific success within this paradigm.

1. Characterizing randomness. Consider a physical process that, if suitably idealized, generates an indefinite sequence of independent random bits. One such process might be radioactive decay of a lump of uranium whose mass is kept at just the level needed to ensure that the probability is one-half that no alpha particle is emitted in the nth microsecond of the experiment. Let us think of the bits as drawn from {0, 1} and denote the resulting sequence by x with coordinates x0, (...) x1, . . .. Now wouldn’t it be odd if there were a computer program P with the following property? (shrink)

Scientific inquiry is represented as a process of rational hypothesis revision in the face of data. For the concept of rationality, we rely on the theory of belief dynamics as developed in [5, 9]. Among other things, it is shown that if belief states are left unclosed under deductive logic then scientific theories can be expanded in a uniform, consistent fashion that allows inquiry to proceed by any method of hypothesis revision based on "kernel" contraction. In contrast, if belief states (...) are closed under logic, then no such expansion is possible. (shrink)

Scientific inquiry is represented as a process of rational hypothesis revision in the face of data. For the concept of rationality, we rely on the theory of belief dynamics as developed in [5, 9]. Among other things, it is shown that if belief states are left unclosed under deductive logic then scientific theories can be expanded in a uniform, consistent fashion that allows inquiry to proceed by any method of hypothesis revision based on “kernel” contraction. In contrast, if belief states (...) are closed under logic, then no such expansion is possible. (shrink)

Establishing reasonable, prior distributions remains a significant obstacle for the construction of probabilistic expert systems. Human assessment of chance is often relied upon for this purpose, but this has the drawback of being inconsistent with axioms of probability. This article advances a method for extracting a coherent distribution of probability from human judgment. The method is based on a psychological model of probabilistic reasoning, followed by a correction phase using linear programming.

We show that standard statistical tests for randomness of finite sequences are language-dependent in an inductively pernicious way.

The volumes are self contained and can be used individually in upper-level undergraduate and graduate courses ranging from introductory psychology, linguistics, ...

A model of inductive inquiry is defined within a first-order context. Intuitively, the model pictures inquiry as a game between Nature and a scientist. To begin the game, a nonlogical vocabulary is agreed upon by the two players along with a partition of a class of structures for that vocabulary. Next, Nature secretly chooses one structure ("the real world") from some cell of the partition. She then presents the scientist with a sequence of atomic facts about the chosen structure. With (...) each new datum the scientist announces a guess about the cell to which the chosen structure belongs. To succeed in his inquiry, the scientist's successive conjectures must be correct all but finitely often, that is, the conjectures must converge in the limit to the correct cell. A special kind of scientist selects his hypotheses on the basis of a belief revision operator. We show that reliance on belief revision allows scientists to solve a wide class of problems. (shrink)

We examine the prospects for finding “best possible” or “ideal” computing machines for various learning tasks. For this purpose, several precise senses of “ideal machine” are considered within the context of formal learning theory. Generally negative results are provided concerning the existence of ideal learning‐machines in the senses considered.

We consider drawbacks to scientific methods that prefer simple hypotheses to complex ones that cover the same data. The discussion proceeds in the context of a precise model of scientific inquiry.