The orthodox view in statistics has it that frequentism and Bayesianism are diametrically opposed—two totally incompatible takes on the problem of statistical inference. This paper argues to the contrary that the two approaches are complementary and need to mesh if probabilistic reasoning is to be carried out correctly.

I will open the first part of this paper by trying to elucidate the frequentist foundations of RCTs. I will then present a number of methodological objections against the viability of these inferential principles in the conduct of actual clinical trials. In the following section, I will explore the main ethical issues in frequentist trials, namely those related to randomisation and the use of stopping rules. In the final section of the first part, I will analyse why RCTs (...) were accepted for regulatory purposes. I contend that their main virtue, from a regulatory viewpoint, is their impartiality, which is grounded in randomisation and fixed rules for the interpretation of the experiment. Thus the question will be whether Bayesian trials can match or exceed the achievements of frequentist RCTs in all these respects. In the second part of the paper, I will first present a quick glimpse of the introduction of Bayesianism in the field of medical experiments, followed by a summary presentation of the basic tenets of a Bayesian trial. The point here is to show that there is no such thing as “a” Bayesian trial. Bayesianism can ground many different approaches to medical experiments and we should assess their respective virtues separately. Thus I present two actual trials, planned with different goals in mind, and assess their respective epistemic, ethical and regulatory merits. In a tentative conclusion, I contend that, given the constraints imposed by our current regulatory framework, impartiality should preside over the design of clinical trials, even at the expense of many of their inferential and ethical virtues. (shrink)

This book analyzes the origins of statistical thinking as well as its related philosophical questions, such as causality, determinism or chance. Bayesian and frequentist approaches are subjected to a historical, cognitive and epistemological analysis, making it possible to not only compare the two competing theories, but to also find a potential solution. The work pursues a naturalistic approach, proceeding from the existence of numerosity in natural environments to the existence of contemporary formulas and methodologies to heuristic pragmatism, a (...) concept introduced in the book's final section. This monograph will be of interest to philosophers and historians of science and students in related fields. Despite the mathematical nature of the topic, no statistical background is required, making the book a valuable read for anyone interested in the history of statistics and human cognition. (shrink)

The debates between Bayesian, frequentist, and other methodologies of statistics have tended to focus on conceptual justifications, sociological arguments, or mathematical proofs of their long run properties. Both Bayesian statistics and frequentist (“classical”) statistics have strong cases on these grounds. In this article, we instead approach the debates in the “Statistics Wars” from a largely unexplored angle: simulations of different methodologies’ performance in the short to medium run. We conducted a large number of simulations using a straightforward (...) decision problem based around tossing a coin with unknown bias and then placing bets. In this simulation, we programmed four players, inspired by Bayesian statistics, frequentist statistics, Jon Williamson’s version of Objective Bayesianism, and a player who simply extrapolates from observed frequencies to general frequencies. The last player functions as a benchmark: a statistical methodology should at least outperform a crude form of induction. We focused on the performance of these methodologies in guiding the players towards good decisions. Unlike an earlier simulation study of this type, we found no systematic difference in performance between the Bayesian and frequentist players, provided the Bayesian used a flat prior and the frequentist used a low confidence level. Unlike that study, we were able to use Big Data methods to mitigate problems of random error in the simulation results. The Williamsonian player, who is a novel element of our study, also had no systematic differences in their performance, provided that they use a low confidence level. These players performed similarly even in the very short run, when players were making different decisions. Our study indicates that all three methodologies should be taken seriously by philosophers and practitioners of statistics. However, the frequentist and Williamsonian performed poorly when their confidence levels were high, and the Bayesian was surprisingly harmed by biased priors, providing some unexpected lessons for these methodologies when facing this type of decision problem. (shrink)

One approach to validating simulation models is to formally compare model outputs with independent data. We consider such model validation from the point of view of Frequentist statistics. A range of estimates and tests of goodness of fit have been advanced. We review these approaches, and demonstrate that some of the tests suffer from difficulties in interpretation because they rely on the null hypothesisHypothesis that the model is similar to the observationsObservations. This reliance creates two unpleasant possibilities, namely, (...) a model could be spuriously validated when dataData are too few, or inappropriately rejected when data are too many. Finally, these tests do not allow a principled declaration of what a reasonable level of difference would be considering the purposes to which the model will be put. We consider equivalence tests, and demonstrate that they do not suffer from the previously identified shortcomings. We provide two case studies to illustrate the claims of the chapter. (shrink)

We analyse the issue of using prior information in frequentist statistical inference. For that purpose, we scrutinise different kinds of sampling designs in Jerzy Neyman’s theory to reveal a variety of ways to explicitly and objectively engage with prior information. Further, we turn to the debate on sampling paradigms (design-based vs. model-based approaches) to argue that Neyman’s theory supports an argument for the intermediate approach in the frequentism vs. Bayesianism debate. We also demonstrate that Neyman’s theory, by allowing (...) non-epistemic values to influence evidence collection and formulation of statistical conclusions, does not compromise the epistemic reliability of the procedures and may improve it. This undermines the value-free ideal of scientific inference. (shrink)

This paper is an extended review of the book Error and Inference, edited by Deborah Mayo and Aris Spanos, about their frequentist and philosophical perspective on testing of hypothesis and on the criticisms of alternatives like the Bayesian approach.

BackgroundDistorted gambling-related cognitions are tightly related to gambling problems, and are one of the main targets of treatment for disordered gambling, but their etiology remains uncertain. Although folk wisdom and some theoretical approaches have linked them to lower domain-general reasoning abilities, evidence regarding that relationship remains unconvincing.MethodIn the present cross-sectional study, the relationship between probabilistic/abstract reasoning, as measured by the Berlin Numeracy Test, and the Matrices Test, respectively, and the five dimensions of the Gambling-Related Cognitions Scale, was tested in a (...) sample of 77 patients with gambling disorder and 58 individuals without gambling problems.Results and interpretationNeither BNT nor matrices scores were significantly related to gambling-related cognitions, according to frequentist analyses, performed both considering and disregarding group in the models. Correlation Bayesian analyses largely supported the null hypothesis, i.e., the absence of relationships between the measures of interest. This pattern or results reinforces the idea that distorted cognitions do not originate in a general lack of understanding of probability or low fluid intelligence, but probably result from motivated reasoning. (shrink)

The goal of this paper is to provide a detailed reading of John Venn's Logic of Chance as a work of logic or, more specifically, as a specific portion of the general system of so-called ‘material’ logic developed in his Principles of Empirical or Inductive Logic and to discuss it against the background of his Boolean-inspired views on the connection between logic and mathematics. It is by means of this situating of Venn 1866 [The Logic of Chance. An Essay on (...) the Foundations and Province of the Theory of Probability. With Especial Reference to Its Application to Moral and Social Science, London: Macmillan] within the entirety of his oeuvre that it becomes both possible to revisit and necessary to re-articulate its place in the history of the frequency interpretation of probability. For it is clear that if Venn's approach to logic not only allowed him to establish its foundations on the basis of a process of idealization and define it as consisting of so-called hypothetical infinite series, bu.. (shrink)

Normal 0 0 1 85 487 UBC 4 1 598 11.773 0 0 0 Under what conditions is the failure to have evidence that p evidence that p is false? Absent evidence reasoning is common in many sciences, including astronomy, archeology, biology and medicine. An often-repeated epistemological motto is that “the absence of evidence is not evidence of absence.” Analysis of absent evidence reasoning usually takes place in a deductive or frequentist hypothesis-testing framework. Instead, I develop a Bayesian analysis (...) of this motto and prove that, under plausible assumptions about the nature of evidence, the absence of evidence is evidence of absence. (shrink)

The data analyzed in this paper are part of the results described in Bueno et al. (2000). Three cytogenetics endpoints were analyzed in three populations of a species of wild rodent – Akodon montensis – living in an industrial, an agricultural, and a preservation area at the Itajaí Valley, State of Santa Catarina, Brazil. The polychromatic/normochromatic ratio, the mitotic index, and the frequency of micronucleated polychromatic erythrocites were used in an attempt to establish a genotoxic profile of each area. It (...) was assumed that the three populations were in the same conditions with respect to the influence of confounding factors such as animal age, health, nutrition status, presence of pathogens, and intra- and inter-populational genetic variability. Therefore, any differences found in the endpoints analyzed could be attributed to the external agents present in each area. The statistical models used in this paper are mixtures of negative-binomials and Poisson variables. The Poisson variables are used as approximations of binomials for rare events. The mixing distributions are beta densities. The statistical analyzes are under the bayesian perspective, as opposed to the frequentist ones often considered in the literature, as for instance in Bueno et al. (2000). (shrink)

This paper makes the case for a wider acceptance of a probabilistic approach to causation in negligence. This acceptance would help to remove much of the incoherence which has come to afflict the English law of personal injury law. This incoherence can also be found in other common law jurisdictions (notably those of the United States, Canada and Australia). Concentrating upon recent UK case law, the argument opposes the contention that ‘naked statistics’ can play no role in establishing causation. (...) The argument is controversial but it can be reduced to three unremarkable grounds: (1) With its acceptance (albeit in certain carefully prescribed circumstances) of liability for a negligently increased risk which has eventuated, the common law has already embraced a probabilistic conception of causation; (2) The English common law already employs a probabilistic (frequentist) approach to identifying coincidences; and (3) With the ‘balance of probabilities’ as the standard of proof in civil cases, the common law has long had a probabilistic (epistemic) concept at its core. Probabilistic approaches (at both the type and token level) are shown to be consistent with laypersons’ understanding of the concepts such as risk, chance, odds and likelihood. Moreover, a wider acceptance of a probabilistic perspective on causation would entail no major challenge to the fundamental aims of tort, viz. deterrence and corrective justice. (shrink)

The way metrologists conceive of measurement has undergone a major shift in the last two decades. This shift can in great part be traced to a change in the statistical methods used to deal with the expression of measurement results, and, more particularly, with the calculation of measurement uncertainties. Indeed, as we show, the incapacity of the frequentist approach to the calculus of uncertainty to deal with systematic errors has prompted the replacement of the customary frequentist methods (...) by fully Bayesian procedures. The epistemological ramifications of the Bayesian approach merge with a deep empiricist mood tantamount to an “epistemic turn”: measurement results are analysed in terms of degrees of belief, and central concepts such as error and accuracy are called into question. We challenge the perspective entailed by this epistemic turn: we insist on the centrality of the concepts of error and accuracy by underlining the intentional character of measurement that is intimately linked to the process of correction of experimental data. We further circumvent the difficulties posed by the classical analysis of measurement by stressing the social rather than the epistemic dimension of measurement activities. (shrink)

The p value is the probability under the null hypothesis of obtaining an experimental result that is at least as extreme as the one that we have actually obtained. That probability plays a crucial role in frequentist statistical inferences. But if we take the word ‘extreme’ to mean ‘improbable’, then we can show that this type of inference can be very problematic. In this paper, I argue that it is a mistake to make such an interpretation. Under minimal assumptions (...) about the alternative hypothesis, I explain why ‘extreme’ means ‘outside the most precise predicted range of experimental outcomes for a given upper bound probability of error’. Doing so, I rebut recent formulations of recurrent criticisms against the frequentist approach in statistics and underscore the importance of random variables. (shrink)

ABSTRACTIn this article, we discuss the benefits of Bayesian statistics and how to utilize them in studies of moral education. To demonstrate concrete examples of the applications of Bayesian statistics to studies of moral education, we reanalyzed two data sets previously collected: one small data set collected from a moral educational intervention experiment, and one big data set from a large-scale Defining Issues Test-2 survey. The results suggest that Bayesian analysis of data sets collected from moral educational studies can provide (...) additional useful statistical information, particularly that associated with the strength of evidence supporting alternative hypotheses, which has not been provided by the classical frequentist approach focusing on P-values. Finally, we introduce several practical guidelines pertaining to how to utilize Bayesian statistics, including the utilization of newly developed free statistical software, Jeffrey’s Amazing Statistics Program, and thresholding based on Bayes Factors, to scholars in the field of moral education. (shrink)

The exponential growth of social data both in volume and complexity has increasingly exposed many of the shortcomings of the conventional frequentist approach to statistics. The scientific community has called for careful usage of the approach and its inference. Meanwhile, the alternative method, Bayesian statistics, still faces considerable barriers toward a more widespread application. The bayesvl R package is an open program, designed for implementing Bayesian modeling and analysis using the Stan language’s no-U-turn (NUTS) sampler. The package (...) combines the ability to construct Bayesian network models using directed acyclic graphs (DAGs), the Markov chain Monte Carlo (MCMC) simulation technique, and the graphic capability of the ggplot2 package. As a result, it can improve the user experience and intuitive understanding when constructing and analyzing Bayesian network models. A case example is offered to illustrate the usefulness of the package for Big Data analytics and cognitive computing. (shrink)

Research in ecology and evolutionary biology (evo-eco) often tries to emulate the “hard” sciences such as physics and chemistry, but to many of its practitioners feels more like the “soft” sciences of psychology and sociology. I argue that this schizophrenic attitude is the result of lack of appreciation of the full consequences of the peculiarity of the evo-eco sciences as lying in between a-historical disciplines such as physics and completely historical ones as like paleontology. Furthermore, evo-eco researchers have gotten stuck (...) on mathematically appealing but philosophi- cally simplistic concepts such as null hypotheses and p-values defined according to the frequentist approach in statistics, with the consequence of having been unable to fully embrace the complexity and subtlety of the problems with which ecologists and evolutionary biologists deal with. I review and discuss some literature in ecology, philosophy of science and psychology to show that a more critical methodological attitude can be liberating for the evo-eco scientist and can lead to a more fecund and enjoyable practice of ecology and evolutionary biology. With this aim, I briefly cover concepts such as the method of multiple hypotheses, Bayesian analysis, and strong inference. (shrink)

The replication crisis has prompted many to call for statistical reform within the psychological sciences. Here we examine issues within Frequentist statistics that may have led to the replication crisis, and we examine the alternative—Bayesian statistics—that many have suggested as a replacement. The Frequentist approach and the Bayesian approach offer radically different perspectives on evidence and inference with the Frequentist approach prioritising error control and the Bayesian approach offering a formal method for quantifying (...) the relative strength of evidence for hypotheses. We suggest that rather than mere statistical reform, what is needed is a better understanding of the different modes of statistical inference and a better understanding of how statistical inference relates to scientific inference. (shrink)

Statistical tests of the primality of some numbers look similar to statistical tests of many nonmathematical, clearly empirical propositions. Yet interpretations of probability prima facie appear to preclude the possibility of statistical tests of mathematical propositions. For example, it is hard to understand how the statement that n is prime could have a frequentist probability other than 0 or 1. On the other hand, subjectivist approaches appear to be saddled with ‘coherence’ constraints on rational probabilities that require rational agents (...) to assign extremal probabilities to logical and mathematical propositions. In the light of these problems, many philosophers have come to think that there must be some way to generalize a Bayesian statistical account. In this article I propose that a classical frequentist approach should be reconsidered. I conclude that we can give a conditional justification of statistical testing of at least some mathematical hypotheses: if statistical tests provide us with reasons to believe or bet on empirical hypotheses in the standard situations, then they also provide us with reasons to believe or bet on mathematical hypotheses in the structurally similar mathematical cases. (shrink)

Evolutionary theory offers the possibility of building an epistemology that requires neither a theory of truth nor a definition of knowledge, thus bypassing some of the more notable difficulties with standard approaches to epistemology. Following a critique of one of the most popular approaches to thinking about cultural evolution I argue for a frequentist approach to evolutionary epistemology, and that cultural transmission should be understood as coordinated phenotypic variability within groups of closely related organisms. I construct a formal (...) system which allows the analysis of the evolution of networks of multiple interacting populations. Of particular epistemological interest is the relationship between genetic evolution and the evolution of cultural entities. The information-theoretic notion of "mutual information" is employed to quantify the transfer of information from the environment as it determines genetic fitnesses into the distribution of acquired traits Two simple models are constructed and analyzed. A two-level model of the navigation of a colony of bacteria demonstrates how selection on inherited traits affects the acquisition of information on the level of transient behavioral traits. A three-level model of bumblebee foraging shows how information about the environment is caught up in distributions of acquired foraging preferences--a bee-analog of human cognitive states. The Appendix contains an analytic proof of information gain under selection. (shrink)

The idea that probability is a degree of rational belief seemed too vague for a foundation of a mathematical theory. It was certainly not obvious that degrees of rational belief had to be governed by the probability axioms as used by Laplace and other prestatistical probabilityst. The axioms seemed arbitrary in their interpretation. To eliminate the arbitrariness, the stat- isticians of the early 20th century drastically restricted the possible applications of the probability theory, by insisting that probabilities had to be (...) interpreted as relative frequencies, which obviously satisfied the probability axioms, and so the arbitrariness was removed. But the frequentist approach turned more subjective than the prestatistical approach, because the identifications of outcome spaces, the choices of test statistics, the declarations of what rejection regions are, the choices of null-hypothesis among alternatives, the contradictory choices between sizes and powers etc., depend on thoughts or even whims of the experimenter. Frequentists thus failed to solve the problems that motivated their approach, they even exacerbated them. The subjective Bayesianism of Ramsey and de Finetti did not solve the problems either. Finally Cox provided the missing foundation for probability as a degree of rational belief, which makes the Bayesian probability theory (which is based on this foundation) the best theory of probable inference we have. Hence, it is quite unbelievable that it is not even mentioned in recent philosophy textbooks devoted to the probable inference. The reason could be that it requires fairly sophisticated mathematics. But not even to mention it? We explain the history and prove Cox theorem in a novel way. (shrink)

Medical research makes intensive use of statistics in order to support its claims. In this paper we make explicit an epistemological tension between the conduct of clinical trials and their interpretation: statistical evidence is sometimes discarded on the basis of an underlined Bayesian reasoning. We suggest that acknowledging the potentiality of Bayesian statistics might contribute to clarify and improve comprehension of medical research. Nevertheless, despite Bayesianism may provide a better account for scientific inference with respect to the standard frequentist (...) approach, Bayesian statistics is rarely adopted in clinical research. The main reason lies in the supposed subjective elements characterizing this perspective. Hence, we discuss this objection presenting the so-called Reference analysis, a formal method which has been developed in the context of objective Bayesian statistics in order to define priors which have a minimal or null impact on posterior probabilities. Furthermore, according to this method only available data are relevant sources of information, so that it resists the most common criticisms against Bayesianism. (shrink)

The objective of the paper is threefold. First, to argue that some of Haavelmo's methodological ideas and insights have been neglected because they are largely at odds with the traditional perspective that views empirical modeling in economics as an exercise in curve-fitting. Second, to make a case that this neglect has contributed to the unreliability of empirical evidence in economics that is largely due to statistical misspecification. The latter affects the reliability of inference by inducing discrepancies between the actual and (...) nominal error probabilities. Third, to argue that the ‘bridge pier’ between theory and data, as envisaged by Haavelmo, T [The Probability Approach to Econometrics, Econometrica, 12, 1–115], can be completed by blending some of his key methodological ideas and insights into a refined/extended version of the frequentist approach to inference. This new perspective is used to revisit dynamic stochastic general equilibrium modeling in an attempt to bring.. (shrink)

We suggest to define objective probabilities by similarity-weighted empirical frequencies, where more similar cases get a higher weight in the computation of frequencies. This formula is justified intuitively and axiomatically, but raises the question, which similarity function should be used? We propose to estimate the similarity function from the data, and thus obtain objective probabilities. We compare this definition to others, and attempt to delineate the scope of situations in which objective probabilities can be used.

Bayes’ rule shows how one might rationally change one’s beliefs in the light of evidence. It is the foundation of a statistical method called Bayesianism. In health care research, Bayesianism has its advocates but the dominant statistical method is frequentism. There are at least two important philosophical differences between these methods. First, Bayesianism takes a subjectivist view of probability (i.e. that probability scores are statements of subjective belief, not objective fact) whilst frequentism takes an objectivist view. Second, Bayesianism is explicitly (...) inductive (i.e. it shows how we may induce views about the world based on partial data from it) whereas frequentism is at least compatible with non-inductive views of scientific method, particularly the critical realism of Popper. Popper and others detail significant problems with induction. Frequentism’s apparent ability to avoid these, plus its ability to give a seemingly more scientific and objective take on probability, lies behind its philosophical appeal to health care researchers. However, there are also significant problems with frequentism, particularly its inability to assign probability scores to single events. Popper thus proposed an alternative objectivist view of probability, called propensity theory, which he allies to a theory of corroboration; but this too has significant problems, in particular, it may not successfully avoid induction. If this is so then Bayesianism might be philosophically the strongest of the statistical approaches. The article sets out a number of its philosophical and methodological attractions. Finally, it outlines a way in which critical realism and Bayesianism might work together. (shrink)

The axiomatization of probability theory by the national socialistic German mathematician E. Tornier will be embedded between measure theoretic and frequentist approaches to the problem and compared with Tomiers ideological claims. New details of his biography will be given.

Frequentism and Bayesianism represent very different approaches to hypothesis testing, and this presents a skeptical challenge for Bayesians. Given that most empirical research uses frequentist methods, why (if at all) should we rely on it? While it is well known that there are conditions under which Bayesian and frequentist methods agree, without some reason to think these conditions are typically met, the Bayesian hasn’t shown why we are usually safe in relying on results reported by significance testers. In (...) this article, I provide arguments that such conditions will usually be met; the Bayesian can maintain her theoretical disagreement with the frequentist while holding that her error is mostly harmless in practice. (shrink)

Frequentist statistical methods continue to predominate in many areas of science despite prominent calls for "statistical reform." They do so in part because their main rivals, Bayesian methods, appeal to prior probability distributions that arguably lack an objective justification in typical cases. Some methodologists find a third approach called likelihoodism attractive because it avoids important objections to frequentism without appealing to prior probabilities. However, likelihoodist methods do not provide guidance for belief or action, but only assessments of data (...) as evidence. I argue that there is no good way to use those assessments to guide beliefs or actions without appealing to prior probabilities, and that as a result likelihoodism is not a viable alternative to frequentism and Bayesianism for statistical reform efforts in science. (shrink)

Cross-situational learning is a mechanism for learning the meaning of words across multiple exposures, despite exposure-by-exposure uncertainty as to the word's true meaning. We present experimental evidence showing that humans learn words effectively using cross-situational learning, even at high levels of referential uncertainty. Both overall success rates and the time taken to learn words are affected by the degree of referential uncertainty, with greater referential uncertainty leading to less reliable, slower learning. Words are also learned less successfully and more slowly (...) if they are presented interleaved with occurrences of other words, although this effect is relatively weak. We present additional analyses of participants’ trial-by-trial behavior showing that participants make use of various cross-situational learning strategies, depending on the difficulty of the word-learning task. When referential uncertainty is low, participants generally apply a rigorous eliminative approach to cross-situational learning. When referential uncertainty is high, or exposures to different words are interleaved, participants apply a frequentist approximation to this eliminative approach. We further suggest that these two ways of exploiting cross-situational information reside on a continuum of learning strategies, underpinned by a single simple associative learning mechanism. (shrink)

The study of cultural evolution has taken on an increasingly interdisciplinary and diverse approach in explicating phenomena of cultural transmission and adoptions. Inspired by this computational movement, this study uses Bayesian networks analysis, combining both the frequentist and the Hamiltonian Markov chain Monte Carlo (MCMC) approach, to investigate the highly representative elements in the cultural evolution of a Vietnamese city’s architecture in the early 20th century. With a focus on the façade design of 68 old houses in (...) Hanoi’s Old Quarter (based on 78 data lines extracted from 248 photos), the study argues that it is plausible to look at the aesthetics, architecture, and designs of the house façade to find traces of cultural evolution in Vietnam, which went through more than six decades of French colonization and centuries of sociocultural influence from China. The in-depth technical analysis, though refuting the presumed model on the probabilistic dependency among the variables, yields several results, the most notable of which is the strong influence of Buddhism over the decorations of the house façade. Particularly, in the top 5 networks with the best Bayesian Information Criterion (BIC) scores and p<0.05, the variable for decorations (DC) always has a direct probabilistic dependency on the variable B for Buddhism. The paper then checks the robustness of these models using Hamiltonian MCMC method and find the posterior distributions of the models’ coefficients all satisfy the technical requirement. Finally, this study suggests integrating Bayesian statistics in the social sciences in general and for the study of cultural evolution and architectural transformation in particular. (shrink)

Classic conceptions of model simplicity for machine learning are mainly based on the analysis of the structure of the model. Bayesian, Frequentist, information theoretic and expressive power concepts are the best known of them, which are reviewed in this work, along with their underlying assumptions and weaknesses. These approaches were developed before the advent of the Big Data deluge, which has overturned the importance of structural simplicity. The computational simplicity concept is presented, and it is argued that it is (...) more encompassing and closer to actual machine learning practices than the classic ones. In order to process the huge datasets which are commonplace nowadays, the computational complexity of the learning algorithm is the decisive factor to assess the viability of a machine learning strategy, while the classic accounts of simplicity play a surrogate role. Some of the desirable features of computational simplicity derive from its reliance on the learning system concept, which integrates key aspects of machine learning that are ignored by the classic concepts. Moreover, computational simplicity is directly associated with energy efficiency. In particular, the question of whether the maximum possibly achievable predictive accuracy should be attained, no matter the economic cost of the associated energy consumption pattern, is considered. (shrink)

This paper aims to investigate the so-called paradox of deterministic probabilities: in a deterministic world, all probabilities should be subjective; however, they also seem to play important explanatory and predictive roles which suggest they are objective. The problem is then to understand what these deterministic probabilities are. Recent proposed solutions of this paradox are the Mentaculus vision, the range account of probability, and a version of frequentism based on typicality. All these approaches aim at defining deterministic objective probabilities as to (...) make them compatible with determinism. In this paper I argue that one can think of the equivalent of subjective and objective deterministic probabilities in terms of typicality. Also, I show that only what I identify with objective probabilities play the necessary explanatory and predictive roles, while the subjective components essentially guide beliefs. In this way, the paradox is solved. (shrink)

This paper outlines the approaches of two apparently competing schools of statistics. The criticisms made by supporters of Bayesian statistics about conventional Frequentist statistics are explained, and the Bayesian claim that their method enables research into new treatments without the need for clinical trials is examined in detail. Several further important issues are considered, including: the use of historical controls and data routinely collected on patients; balance in randomised trials; the possibility of giving information to patients; patient choice and (...) patient autonomy; and how widely the results of clinical trials can be used. It is concluded that good statistical techniques in the design and analysis of medical studies are essential, but the statistical school used in developing such techniques is relatively unimportant. (shrink)

We have been oversold on the base rate fallacy in probabilistic judgment from an empirical, normative, and methodological standpoint. At the empirical level, a thorough examination of the base rate literature (including the famous lawyer–engineer problem) does not support the conventional wisdom that people routinely ignore base rates. Quite the contrary, the literature shows that base rates are almost always used and that their degree of use depends on task structure and representation. Specifically, base rates play a relatively larger role (...) in tasks where base rates are implicitly learned or can be represented in frequentist terms. Base rates are also used more when they are reliable and relatively more diagnostic than available individuating information. At the normative level, the base rate fallacy should be rejected because few tasks map unambiguously into the narrow framework that is held up as the standard of good decision making. Mechanical applications of Bayes's theorem to identify performance errors are inappropriate when (1) key assumptions of the model are either unchecked or grossly violated, and (2) no attempt is made to identify the decision maker's goals, values, and task assumptions. Methodologically, the current approach is criticized for its failure to consider how the ambiguous, unreliable, and unstable base rates of the real world are and should be used. Where decision makers' assumptions and goals vary, and where performance criteria are complex, the traditional Bayesian standard is insufficient. Even where predictive accuracy is the goal in commonly defined problems, there may be situations (e.g., informationally redundant environments) in which base rates can be ignored with impunity. A more ecologically valid research program is called for. This program should emphasize the development of prescriptive theory in rich, realistic decision environments. (shrink)

In the curve fitting problem two conflicting desiderata, simplicity and goodness-of-fit pull in opposite directions. To solve this problem, two proposals, the first one based on Bayes's theorem criterion (BTC) and the second one advocated by Forster and Sober based on Akaike's Information Criterion (AIC) are discussed. We show that AIC, which is frequentist in spirit, is logically equivalent to BTC, provided that a suitable choice of priors is made. We evaluate the charges against Bayesianism and contend that AIC (...) approach has shortcomings. We also discuss the relationship between Schwarz's Bayesian Information Criterion and BTC. (shrink)

Statisticians in medicine can disagree on appropriate methodology applicable to the design and analysis of clinical trials. So called Bayesians and frequentists both claim ethical superiority. This paper, by defining and then linking together various dichotomies, argues there is a place for both statistical camps. The choice between them depends on the phase of clinical trial, disease prevalence and severity, but supremely on the ethics underlying the particular trial. There is always a tension present between physicians primarily obligated to their (...) own patients (the weight of 'individual ethics') and ethical committees responsible for the scientific merit of the trial and its long-term implications ('collective ethics'). Individual ethics, it is proposed, favour the Bayesian approach; collective ethics, the frequentist. Though in some situations the choice appears clear-cut, there remain other where both methodologies can be appropriate. (shrink)

The field of psychology, including cognitive science, is vexed by a crisis of confidence. Although the causes and solutions are varied, we focus here on a common logical problem in inference. The default mode of inference is significance testing, which has a free lunch property where researchers need not make detailed assumptions about the alternative to test the null hypothesis. We present the argument that there is no free lunch; that is, valid testing requires that researchers test the null against (...) a well-specified alternative. We show how this requirement follows from the basic tenets of conventional and Bayesian probability. Moreover, we show in both the conventional and Bayesian framework that not specifying the alternative may lead to rejections of the null hypothesis with scant evidence. We review both frequentist and Bayesian approaches to specifying alternatives, and we show how such specifications improve inference. The field of cognitive science will benefit because consideration of reasonable alternatives will undoubtedly sharpen the intellectual underpinnings of research. (shrink)

Conventional discussion of research methodology contrast two approaches, the quantitative and the qualitative, presented as collectively exhaustive. But if qualitative is taken as the understanding of lifeworlds, the two approaches between them cover only a tiny fraction of research methodologies; and the quantitative, taken as the routine application to controlled experiments of frequentist statistics by way of the Null Hypothesis Significance Testing Procedure, is seriously flawed. It is contrary to the advice both of Fisher and of Neyman and Pearson, (...) the two sources from which it is drawn. (shrink)

Conventional discussion of research methodology contrast two approaches, the quantitative and the qualitative, presented as collectively exhaustive. But if qualitative is taken as the understanding of lifeworlds, the two approaches between them cover only a tiny fraction of research methodologies; and the quantitative, taken as the routine application to controlled experiments of frequentist statistics by way of the Null Hypothesis Significance Testing Procedure, is seriously flawed. It is contrary to the advice both of Fisher and of Neyman and Pearson, (...) the two sources from which it is drawn. (shrink)

We here report the findings of our investigation into the validity of the conjunction fallacy (Tversky & Kahneman, 1983), bearing in mind the role of conversational rules. Our first experiment showed that subjects found a logically correct answer unacceptable when it implied a violation of the conversational rules. We argue that tautological questions, such as those which concern the relationship of inclusion between a class and its sub-class, violate conversational rules because they are not informative. In this sense, it is (...) not understood that the question in a Linda-type problem involves a comparison between an inclusive and included class, but presumed that a different type of comparison is intended. Tautological questions (and, consequently, also their answers) do not become a matter of discussion except under certain specific conditions. Our second experiment showed that, providing the context was adequately marked (such as in the case of a rhetorical question), the conjunction fallacy disappears. In two further experiments, the implications of our view were compared with those of the other critical approaches to the heuristic programme: the classical pragmatic view (which we call logical complementarity) and the frequentist view. (shrink)

The aim of this article is twofold. First, we shall review and analyse the neo-kantian justification for the application of probabilistic concepts in science that was defended by Hans Reichenbach early in his career, notably in his dissertation of 1916. At first sight this kantian approach seems to contrast sharply with Reichenbach’s later logical positivist, frequentist viewpoint. But, and this is our second goal, we shall attempt to show that there is an underlying continuity in Reichenbach’s thought: typical (...) features of his early kantian conceptions can still be recognized in his later work. (shrink)

This paper deals with meta-statistical questions concerning frequentist statistics. In Sections 2 to 4 I analyse the dispute between Fisher and Neyman on the so called logic of statistical inference, a polemic that has been concomitant of the development of mathematical statistics. My conclusion is that, whenever mathematical statistics makes it possible to draw inferences, it only uses deductive reasoning. Therefore I reject Fisher's inductive approach to the statistical estimation theory and adhere to Neyman's deductive one. On the (...) other hand, I assert that Neyman-Pearson's testing theory, as well as Fisher's tests of significance, properly belong to decision theory, not to logic, neither deductive nor inductive. I then also disagree with Costantini's view of Fisher's testing model as a theory of hypothetico-deductive inferences.In Section 5 I disapprove Hacking1's evidentialists criticisms of the Neyman-Pearson's theory of statistics (NPT), as well as Hacking2's interpretation of NPT as a theory of probable inference. In both cases Hacking misses the point. I conclude, by claiming that Mayo's conception of the Neyman-Pearson's testing theory, as a model of learning from experience, does not purport any advantages over Neyman's behavioristic model. (shrink)

The interpretation of tests of a point null hypothesis against an unspecified alternative is a classical and yet unresolved issue in statistical methodology. This paper approaches the problem from the perspective of Lindley's Paradox: the divergence of Bayesian and frequentist inference in hypothesis tests with large sample size. I contend that the standard approaches in both frameworks fail to resolve the paradox. As an alternative, I suggest the Bayesian Reference Criterion: it targets the predictive performance of the null hypothesis (...) in future experiments; it provides a proper decision-theoretic model for testing a point null hypothesis and it convincingly accounts for Lindley's Paradox. (shrink)