Can the evidence provided by software systems meet the standard of proof for civil or criminal cases, and is it individualized evidence? Or, to the contrary, do software systems exclusively provide bare statistical evidence? In this paper, we argue that there are cases in which evidence in the form of probabilities computed by software systems is not bare statistical evidence, and is thus able to meet the standard of proof. First, based on the (...) case of State v. Loomis, we investigate recidivism predictions provided by software systems used in the courtroom. Here, we raise problems for software systems that provide predictions that are based on bare statistical evidence. Second, by examining the case of People v. Chubbs, we argue that the statistical evidence provided by software systems in cold hit DNA cases may in some cases suffice for individualized evidence, on a view on which individualized evidence is evidence that normically supports the relevant proposition (Smith, in Mind 127:1193–1218, 2018). (shrink)

Philosophical debates over statistical evidence have long been framed and dominated by L. Jonathan Cohen's Paradox of the Gatecrasher and a related hypothetical example commonly called Prison Yard. These examples, however, raise an issue not discussed in the large and growing literature on statistical evidence – the question of what statistical evidence is supposed to be evidence of. In actual practice, the legal system does not start with a defendant and then attempt to (...) determine if that defendant has committed some unspecified or under-specified act, as these examples appear to suppose. Rather, both criminal and civil litigation start with a sufficiently specified act and then attempt to determine if the defendant has committed it. And when we start with a more fully specified act, the statistics look very different, and these prominent examples no longer present the paradox they are claimed to support. Examining the issue of specification, however, does more than simply undercut the prominent examples in a long and extensive literature. The examination also raises normative issues challenging the legal system's traditional reluctance to base liability on the conjunction of probabilities. (shrink)

The use of algorithms to support prediction-based decision-making is becoming commonplace in a range of domains including health, criminal justice, education, social services, lending, and hiring. An assumption governing such decisions is that there is a property Y such that individual a should be allocated resource R by decision-maker D if a is Y. When there is uncertainty about whether a is Y, algorithms may provide valuable decision support by accurately predicting whether a is Y on the basis of known (...) features of a. Based on recent work on statistical evidence in epistemology this article presents an argument against relying exclusively on algorithmic predictions to allocate resources when they provide purely statistical evidence that a is Y. The article then responds to the objection that any evidence that will increase the proportion of correct decisions should be accepted as the basis for allocations regardless of its epistemic deficiency. Finally, some important practical aspects of the conclusion are considered. (shrink)

Many who think that naked statistical evidence alone is inadequate for a trial verdict think that use of probability is the problem, and something other than probability – knowledge, full belief, causal relations – is the solution. I argue that the issue of whether naked statistical evidence is weak can be formulated within the probabilistic idiom, as the question whether likelihoods or only posterior probabilities should be taken into account in our judgment of a case. This (...) question also identifies a major difference between the Process Reliabilist and Probabilistic Tracking views of knowledge and other concepts. Though both are externalist, and probabilistic, epistemic theories, Tracking does and Process Reliabilism does not put conditions on likelihoods. So Tracking implies that a naked statistic is not adequate evidence about an individual, and does not yield knowledge, whereas the Reliabilist thinks it gives justified belief and knowledge. Not only does the Tracking view imply that naked statistical evidence is insufficient for a verdict, but it gives us resources to explain why, in terms of risk and the special conditions of a trial. (shrink)

Statistical evidence is pervasive in medicine. In this chapter we will focus on the reliability of randomized clinical trials (RCTs) conducted to test the safety and efficacy of medical treatments. RCTs are scientific experiments and, as such, we expect them to be replicable: if we repeat the same experiment time and again, we should obtain the same outcome (Norton 2015). The statistical design of the test should guarantee that the observed outcome is not a random event, but (...) rather a real effect of the treatments administered. However, for more than a decade now, we have been discussing a replicability crisis across different experimental disciplines including medicine: the outcomes of trials published in very prestigious journals often disappear when the experiment is repeated (see, e.g., Lehrer 2010, Begley and Ellis 2012, Horton 2015). -/- There are different accounts of the reason for this replicability crisis, ranging from scientific fraud to lack of institutional incentives to double-check someone else’s results. In this chapter we will use the replicability crisis as a thread to introduce some central issues in the design of scientific experiments in medicine. First, in section 1, we will see how replicability and statistical significance are connected: we can only make sense of the p-value of a trial outcome within a series of replications of the test. But in order to conduct these replications properly, we need to agree on the proper design of the experiment we are going to repeat. In particular, we need to prevent the preferences of the experimenters from biasing the outcome of the experiment. If there is such a bias, when the experiment is replicated by a third party, the observed outcome will vanish. In section 2, we will argue that trialists need to agree on the debiasing procedures and the statistical quality controls that feature in the trial protocol, if they want the outcome to be replicable. In section 3, we will make two complementary points. On the one hand, replicability per se is not everything: we need trial outcomes that are not only statistically significant but also clinically relevant. On the other hand, trials are not everything: the experts analyzing the evidence can improve the reliability of statistical evidence, although they sometimes fail; we need to further study how they make their decisions. In section 4, we will use a controversy about the overprescription of statins to show how non-replicable effects are obtained in trials and how experts may fail at detecting such flaws, if the commercial interests are big enough. (shrink)

Epistemologists have posed the following puzzle, known as the proof paradox: Why is it intuitively problematic for juries to convict on the basis of statistical evidence and yet intuitively unproblematic for juries to convict on the basis of far less reliable, non-statistical evidence? To answer this question, theorists have explained the exclusion of statistical evidence by arguing that legal proof requires certain epistemic features. In this paper, I make two contributions to the debate. First, (...) I establish the Criminal Verdict Asymmetry, a previously-unarticulated asymmetry between the epistemic norms of guilty and not guilty verdicts. I argue that the prosecution and defense’s different epistemic burdens influence whether statistical evidence can generate the type of verdict each side pursues. Second, I point out a mistake in how theorists have understood the role of statistical evidence in criminal trials. Though epistemologists have primarily focused on whether statistical evidence can generate specific epistemic features required for convictions, I consider whether statistical evidence can demonstrate a lack of such features. I find that there are epistemic advantages to allowing the defense to introduce statistical evidence which can reveal the prosecution’s failure to prove the defendant’s guilt. (shrink)

Courts and jurors sometimes refuse to assign liability to defendants on the basis of statistics alone, despite their apparent reliability. I argue that this refusal is best understood as a recognition of defendants’ right to security. Understood as a robust good in Philip Pettit’s sense, security requires that someone risking harm to others’ protected interests adopt a disposition of concern that controls against wrongfully harming them. Since trials risk harm, the state must adopt such a disposition. Statistics leave open the (...) possibility of innocence and so wrongful harm in a way that the state cannot ignore; if it does use such statistics, it makes defendants wrongfully insecure. This explanation of the problem of bare statistical evidence is especially apt because security grounds the right to a fair trial and judicial procedural rights more generally. (shrink)

Philosophical Issues, Volume 31, Issue 1, Page 128-145, October 2021.

In his recent monograph [7], Professor Shafer has offered us an alternative to Bayesian inference with his novel theory of belief functions and, in his current paper [8], has characterized his position by pointing to two basic differences it shares with Bayesianism. First, belief functions are non-additive so that the degree of belief assigned to the disjunction ‘A1 or A2’ may be larger than the sum of the degrees of belief assigned to the separate disjuncts. Second, the theory of belief (...) functions has its own rule for determining the commitments to changes in degrees of belief when evidence is compounded. So, instead of the Bayesian postulate of conditionalization, that is, in place of using Bayes’ theorem to identify the commitments to changes in probability when evidence accumulates, the theory advocated by Professor Shafer relies on a proposal he traces to A.P.Dempster, which he calls Dempster’s rule for combination of belief functions. (shrink)

PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association, Vol. 1978, Volume Two: Symposia and Invited Papers. (1978), pp. 478-489.

The rapidly increasing role of automation throughout the economy, culture and our personal lives has generated a large literature on the risks of algorithmic decision-making, particularly in high-stakes legal settings. Algorithmic tools are charged with bias, shrouded in secrecy, and frequently difficult to interpret. However, these criticisms have tended to focus on particular implementations, specific predictive techniques, and the idiosyncrasies of the American legal-regulatory regime. They do not address the more fundamental unease about the prospect that we might one day (...) replace judges with algorithms, no matter how fair, transparent, and intelligible they become. The aim of this paper is to propose an account of the source of that unease, and to evaluate its plausibility. I trace foundational unease with algorithmic decision-making in the law to the powerful intuition that there is a basic moral and legal difference between showing that something is true of many people just like you and showing that it is true of you. Human judgment attends to the exception; automation insists on blindly applying the rule. I show how this intuitive thought is connected to both epistemological arguments about the value of statistical evidence, as well as to court-centered conceptions of the rule of law. Unease with algorithmic decision-making in the law thus draws on an intuitive principle that underpins a disparate range of views in legal philosophy. This suggests the principle is deeply ingrained. Nonetheless, I argue that the powerful intuition is not as decisive as it may seem, and indeed runs into significant epistemological and normative challenges. At an epistemological level, I show how concerns about statistical evidence's ability to track the truth can be resolved by adopting a probabilistic, rather than modal, conception of truth-tracking. At a normative level, commitment to highly individualized decision-making co-exists with equally ingrained and competing principles, such as consistent application of law. This suggests that the “rule of law” may not identify a discrete set of institutional arrangements, as proponents of a court-centric conception would have it, but rather a more loosely defined set of values that could potentially be operationalized in multiple ways, including through some level of algorithmic adjudication. Although the prospect of replacing judges with algorithms is indeed unsettling, it does not necessarily entail unreasonable verdicts or an attack on the rule of law. (shrink)

In responding to Prof. Allen's paper, I make several general methodological points: about the use of hypothetical cases, about the point of theorizing, and about the role of idealization. Then I make some more specific points about his claims about (and against) previous work on statistical evidence.

The law views with suspicion statistical evidence, even evidence that is probabilistically on a par with direct, individual evidence that the law is in no way suspicious of. But it has proved remarkably hard to either justify this suspicion, or to debunk it. In this paper, we connect the discussion of statistical evidence to broader epistemological discussions of similar phenomena. We highlight Sensitivity – the requirement that a belief be counterfactually sensitive to the truth (...) in a specific way – as a way of epistemically explaining the legal suspicion towards statistical evidence. Still, we do not think of this as a satisfactory vindication of the reluctance to rely on statistical evidence. Knowledge – and Sensitivity, and indeed epistemology in general – are of little, if any, legal value. Instead, we tell an incentive-based story vindicating the suspicion towards statistical evidence. We conclude by showing that the epistemological story and the incentive-based story are closely and interestingly related, and by offering initial thoughts about the role of statistical evidence in morality. (shrink)

To whatever extent the use of a behavioral, not an evidential, interpretation of decisions in the Lindley-Savage argument for Bayesian theory undermines its cogency as a criticism of typical standard practice, it also undermines the Neyman-Pearson theory as a support for typical standard practice. This leaves standard practice with far less theoretical support than Bayesian methods. It does nothing to resolve the anomalies and paradoxes of standard methods. (Similar statements apply to the common protestation that the models are not real (...) anyway.) The appropriate interpretation of tests as evidence, if possible at all, is difficult and counterintuitive. Any attempt to support tests as more than rules of thumb is doomed to failure. (shrink)

Martin Smith has recently proposed, in this journal, a novel and intriguing approach to puzzles and paradoxes in evidence law arising from the evidential standard of the Preponderance of the Evidence. According to Smith, the relation of normic support provides us with an elegant solution to those puzzles. In this paper I develop a counterexample to Smith’s approach and argue that normic support can neither account for our reluctance to base affirmative verdicts on bare statistical evidence (...) nor resolve the pertinent paradoxes. Normic support is, as a consequence, not a successful epistemic anti-luck condition. (shrink)

It is one thing to hold that merely statistical evidence is _sometimes_ insufficient for rational belief, as in typical lottery and profiling cases. It is another thing to hold that merely statistical evidence is _always_ insufficient for rational belief. Indeed, there are cases where statistical evidence plainly does justify belief. This project develops a dispositional account of the normativity of statistical evidence, where the dispositions that ground justifying statistical evidence are (...) connected to the goals (= proper function) of objects. There are strong intuitive motivations for doing this. For we can turn almost any case of _non-justifying_ merely statistical evidence into a case of _justifying_ merely statistical evidence by adding information about the dispositions and goals of the objects involved. The resulting view not only helps us understand when and why merely statistical evidence is normatively significant, but it also helps us understand how statistical evidence relates to more standard forms of evidence (perceptual, testimonial). The emerging view also has surprising applications, as it imposes limitations on the epistemic value of fine-tuning arguments for theism as well as undermines a standard class of case-based arguments for moral encroachment. (shrink)

The paper takes a closer look at the role of knowledge and evidence in legal theory. In particular, the paper examines a puzzle arising from the evidential standard Preponderance of the Evidence and its application in civil procedure. Legal scholars have argued since at least the 1940s that the rule of the Preponderance of the Evidence gives rise to a puzzle concerning the role of statistical evidence in judicial proceedings, sometimes referred to as the Problem (...) of Bare Statistical Evidence. While this puzzle has led to the development of a multitude of accounts and approaches in the legal literature, I argue here that the problem can be resolved fairly straightforwardly within a knowledge-first framework. (shrink)

This essay is an accessible introduction to the proof paradox in legal epistemology. -/- In 1902 the Supreme Judicial Court of Maine filed an influential legal verdict. The judge claimed that in order to find a defendant culpable, the plaintiff “must adduce evidence other than a majority of chances”. The judge thereby claimed that bare statistical evidence does not suffice for legal proof. -/- In this essay I first motivate the claim that bare statistical evidence (...) does not suffice for legal proof. I then introduce and motivate a knowledge-centred explanation of this fact. The knowledge-centred explanation rests on two premises. The first is that legal proof requires knowledge of culpability. The second is that one cannot attain knowledge that p from bare statistical evidence that p. To motivate the second premise, I suggest that beliefs based on bare statistical evidence fail to be safe—they could easily be wrong—and bare statistical evidence cannot eliminate relevant alternatives. -/- I then cast doubt on the first premise; I argue that legal proof does not require knowledge. I thereby dispute the knowledge-centred explanation of the inadequacy of bare statistical evidence for legal proof. Instead of appealing to the nature of knowledge, I suggest we should seek a more direct explanation by appealing to those more foundational epistemic properties, such as safety or eliminating relevant alternatives. (shrink)

Philosophers subscribing to particular principles of statistical inference and evidence need to be aware of the limitations and practical consequences of the statistical approach they endorse. The framework proposed (for statistical inference in the field of medicine) allows disparate statistical approaches to emerge in their appropriate context. My dissertation proposes a decision theoretic model, together with methodological guidelines, that provide important considerations for deciding on clinical trial conduct. These considerations do not amount to more stopping (...) rules. Instead, they are principles that address the complexity of interpreting and responding to interim data, based on a broad range of epistemic and ethical factors. While they are not stopping rules, they would assist a Data Monitoring Committee in judging its position with regard to necessary precautionary interpretation of interim data. By vindicating a framework that accommodates a wide range of approaches to statistical inference in one important setting (clinical trials), my results pose a serious challenge for any approach that advocates a single, universal principle of statistical inference. (shrink)

As health care embraces the tenets of evidence-based medicine it is important to ask questions about how evidence is produced and interpreted. This essay explores normative dimensions of evidence production, particularly around issues of setting the tolerable level of uncertainty of results. Four specific aspects are explored: what health care providers know about statistics, why alpha levels have been set at 0.05, the role of randomization in the generation of sufficient grounds of belief, and the role of (...) observational studies. The essay concludes with recommendations to acknowledge the value permeation of outcome measures and suggests that attention to reasoning and argument analysis can augment traditional evidence-based approaches in providing a robust critical approach to medical knowledge. (shrink)

With special attention being paid to recent developments in Artificial Intelligence and the Law, specifically related to evidentiary reasoning, this book ...

Recently, the practice of deciding legal cases on purely statistical evidence has been widely criticised. Many feel uncomfortable with finding someone guilty on the basis of bare probabilities, even though the chance of error might be stupendously small. This is an important issue: with the rise of DNA profiling, courts are increasingly faced with purely statistical evidence. A prominent line of argument—endorsed by Blome-Tillmann 2017; Smith 2018; and Littlejohn 2018—rejects the use of such evidence by (...) appealing to epistemic norms that apply to individual inquirers. My aim in this paper is to rehabilitate purely statistical evidence by arguing that, given the broader aims of legal systems, there are scenarios in which relying on such evidence is appropriate. Along the way I explain why popular arguments appealing to individual epistemic norms to reject legal reliance on bare statistics are unconvincing, by showing that courts and individuals face different epistemic predicaments (in short, individuals can hedge when confronted with statistical evidence, whilst legal tribunals cannot). I also correct some misconceptions about legal practice that have found their way into the recent literature. (shrink)

Recent attempts to resolve the Paradox of the Gatecrasher rest on a now familiar distinction between individual and bare statistical evidence. This paper investigates two such approaches, the causal approach to individual evidence and a recently influential (and award-winning) modal account that explicates individual evidence in terms of Nozick's notion of sensitivity. This paper offers counterexamples to both approaches, explicates a problem concerning necessary truths for the sensitivity account, and argues that either view is implausibly committed (...) to the impossibility of no-fault wrongful convictions. The paper finally concludes that the distinction between individual and bare statistical evidence cannot be maintained in terms of causation or sensitivity. We have to look elsewhere for a solution of the Paradox of the Gatecrasher. (shrink)

According to the Rational Threshold View, a rational agent believes p if and only if her credence in p is equal to or greater than a certain threshold. One of the most serious challenges for this view is the problem of statistical evidence: statistical evidence is often not sufficient to make an outright belief rational, no matter how probable the target proposition is given such evidence. This indicates that rational belief is not as sensitive to (...) statistical evidence as rational credence. The aim of this paper is twofold. First, we argue that, in addition to playing a decisive role in rationalizing outright belief, non-statistical evidence also plays a preponderant role in rationalizing credence. More precisely, when both types of evidence are present in a context, non-statistical evidence should receive a heavier weight than statistical evidence in determining rational credence. Second, based on this result, we argue that a modified version of the Rational Threshold View can avoid the problem of statistical evidence. We conclude by suggesting a possible explanation of the varying sensitivity to different types of evidence for belief and credence based on the respective aims of these attitudes. (shrink)

Discrimination law is a possible application of the methods of causal modelling. With it, it brings the possibility of direct statistical evidence on counterfactual questions, something that traditional techniques like multiple regression lack. The kinds of evidence that causal modelling can provide, in large part due to its attention to counterfactuals, is very close to the key question that we ask of jurors in discrimination cases. With this new kind of evidence comes new opportunities. We can (...) better proportion punitive damages to the severity of the discrimination that manifests in a hiring process. We can avoid making certain kinds of assumptions regarding the relationship between protected classes and hiring qualifications that other statistical methods demand from statisticians. We can also distribute restitution to individual claimants in a way that is proportionate to how their application was treated in the hiring process. Here we explore where and how causal modelling can be useful in discrimination law and policy. What elements of law provide friction with this mode of gathering statistical evidence, what new possibilities does it reveal, and how does this integrate with prior judgments regarding statistical evidence? (shrink)

Do privacy rights restrict what is permissible to infer about others based on statistical evidence? This paper replies affirmatively by defending the following symmetry: there is not necessarily a morally relevant difference between directly appropriating people’s private information—say, by using an X-ray device on their private safes—and using predictive technologies to infer the same content, at least in cases where the evidence has a roughly similar probative value. This conclusion is of theoretical interest because a comprehensive justification (...) of the thought that statistical inferences can violate privacy rights is lacking in the current literature. Secondly, the conclusion is of practical interest due to the need for moral assessment of emerging predictive algorithms. (shrink)

Many philosophers have pointed out that statistical evidence, or at least some forms of it, lack desirable epistemic or non-epistemic properties, and that this should make us wary of litigations in which the case against the defendant rests in whole or in part on statistical evidence. Others have responded that such broad reservations about statistical evidence are overly restrictive since appellate courts have expressed nuanced views about statistical evidence. In an effort to (...) clarify and reconcile, I put forward an interpretive analysis of why statistical evidence should raise concerns in some cases but not others. I argue that when there is a mismatch between the specificity of the evidence and the expected specificity of the accusation, statistical evidence—as any other kind of evidence—should be considered insufficient to sustain a conviction. I rely on different stylized court cases to illustrate the explanatory power of this analysis. (shrink)

Abstract. Suppose that the word of an eyewitness makes it 80% probable that A committed a crime, and that B is drawn from a population in which the incidence rate of that crime is 80%. Many philosophers and legal theorists have held that if this is our only evidence against those parties then (i) we may be justified in finding against A but not against B; but (ii) that doing so incurs a loss in the accuracy of our findings. (...) This paper argues against (ii). It argues that accuracy considerations can motivate taking different attitudes towards individualized and statistical evidence even across cases where they generate the same probability that the defendant is guilty. (shrink)

Whenever a litigant needs to prove that a certain result was caused in a specific way, what could be more compelling than citing the infinitesimal probability of that result emanating from an alternative natural cause? Contrary to this intuitive position, in the present article, I argue that the contention that a result was due to a certain cause should remain unaffected by statistical evidence of the extremely low probability of an alternative cause. The only scenario in which the (...) low probability of a natural cause would be relevant to the case at hand is if it were contrasted with another piece of statistical evidence: the frequency of the criminal activity among people who are similar to the accused. However, by connecting the use of probabilistic generalisations in legal fact-finding to the issue of free will, I hold that, in Criminal Law, contrasting frequencies in this manner is objectionable—as a matter of principle—regardless of how reliable the statistical analysis is. Consequently, if the low probability of a natural cause is probative only if contrasted with another piece of statistical evidence that is objectionable, then neither piece of evidence should be admitted in criminal trials. (shrink)

While many philosophers of science have accorded special evidential significance to tests whose results are "novel facts", there continues to be disagreement over both the definition of novelty and why it should matter. The view of novelty favored by Giere, Lakatos, Worrall and many others is that of use-novelty: An accordance between evidence e and hypothesis h provides a genuine test of h only if e is not used in h's construction. I argue that what lies behind the intuition (...) that novelty matters is the deeper intuition that severe tests matter. I set out a criterion of severity akin to the notion of a test's power in Neyman-Pearson statistics. I argue that tests which are use-novel may fail to be severe, and tests that are severe may fail to be use-novel. I discuss the 1919 eclipse data as a severe test of Einstein's law of gravity. (shrink)

In order to perform certain actions – such as incarcerating a person or revoking parental rights – the state must establish certain facts to a particular standard of proof. These standards – such as preponderance of evidence and beyond reasonable doubt – are often interpreted as likelihoods or epistemic confidences. Many theorists construe them numerically; beyond reasonable doubt, for example, is often construed as 90 to 95% confidence in the guilt of the defendant. -/- A family of influential cases (...) suggests standards of proof should not be interpreted numerically. These ‘proof paradoxes’ illustrate that purely statistical evidence can warrant high credence in a disputed fact without satisfying the relevant legal standard. In this essay I evaluate three influential attempts to explain why merely statistical evidence cannot satisfy legal standards. (shrink)

Classical statistical inferences have been criticised for various reasons. To assess the soundness of such criticisms is a very important task because they are widely used in everyday scientific research. This is one of the reasons why the philosophy of statistics is an exciting field of study. In this paper, I focus on two such criticisms. The first one claims that the use of the p-value violates the principle of total evidence. It is a thesis that has been (...) defended by Elliott Sober and Bengt Autzen. The second one says that the result of classical tests does not only depend on the data but on the sampling plan of the experimenter also. The underlying criticism of course is that the sampling plan is not part of the evidence and that classical tests therefore violate PTE. The intentions of the experimenter should not affect the result of an inference. My aim is to show that both criticisms are unsound. Doing so, I hope to clarify the concept of p-value and the nature of the evidence in classical statistical tests. The point of my paper is to show that the identification of the evidence on which those criticisms rest is inadequate. (shrink)

Patient preference predictors (PPPs) promise to provide medical professionals with a new solution to the problem of making treatment decisions on behalf of incapacitated patients. I show that the use of PPPs faces a version of a normative problem familiar from legal scholarship: the problem of naked statistical evidence. I sketch two sorts of possible reply, vindicating and debunking, and suggest that our reply to the problem in the one domain ought to mirror our reply in the other. (...) The conclusion is thus conditional: if we think the problem of naked statistical evidence is a serious problem in the legal domain, then we should be concerned about the symmetrical problem for PPPs. (shrink)

Hacking’s Law of Likelihood says—paraphrasing—that data support hypothesis H1 over hypothesis H2 whenever the likelihood ratio for H1 over H2 exceeds 1. But Hacking later noted a seemingly fatal flaw in the LR itself: it cannot be interpreted as the degree of “evidential significance” across applications. I agree with Hacking about the problem, but I do not believe the condition is incurable. I argue here that the LR can be properly calibrated with respect to the underlying evidence, and I (...) sketch the rudiments of a methodology for so doing. (shrink)

The past two decades have seen a dramatic growth in the use of statisticians and economists for the presentation of expert testimony in legal proceedings. In this paper, we describe a hypothetical case modeled on real ones and involving statistical testimony regarding the causal effect of lead on lowering the IQs of children who ingest lead paint chips. The data we use come from a well-known pioneering study on the topic and the analyses we describe as the expert testimony (...) are similar to ones that can be found in major scientific journals. The battle of the experts in this hypothetical case resembles that which many encounter as expert witnesses. The paper concludes with some observations and advice. (shrink)

This article examines Smith’s recent treatment of the problem of statistical evidence and the conception of epistemic justification that he puts forward. Two possible solutions to the problem of statistical evidence that result from his analysis of cases involving a contrast between statistical and individual evidence are considered. The solution resulting from Smith’s conception of epistemic justification is shown to be inferior to the solution calling for an explanationist conception of epistemic justification. As a (...) result, Smith’s analysis of cases illustrating the problem of statistical evidence, given his own standards of theory choice, is shown to support an explanationist conception of epistemic justification as an alternative to the probabilistic conception that he urges us to reject. (shrink)

The poverty of stimulus argument is one of the most controversial arguments in the study of language acquisition. Here we follow previous approaches challenging the assumption of impoverished primary linguistic data, focusing on the specific problem of auxiliary (AUX) fronting in complex polar interrogatives. We develop a series of corpus analyses of child-directed speech showing that there is indirect statistical information useful for correct auxiliary fronting in polar interrogatives and that such information is sufficient for distinguishing between grammatical and (...) ungrammatical generalizations, even in the absence of direct evidence. We further show that there are simple learning devices, such as neural networks, capable of exploiting such statistical cues, producing a bias toward correct AUX questions when compared to their ungrammatical counterparts. The results suggest that the basic assumptions of the poverty of stimulus argument may need to be reappraised. (shrink)

The lottery problem is the problem of explaining why mere reflection on the long odds that one will lose the lottery does not yield knowledge that one will lose. More generally, it is the problem of explaining why true beliefs merely formed on the basis of statistical evidence do not amount to knowledge. Some have thought that the lottery problem can be solved by appeal to a violation of the safety principle for knowledge, i.e., the principle that if (...) S knows that p, not easily would S have believed that p without p being the case. Against the standard safety‐based solution, I argue that understanding safe belief as belief that directly covaries with the truth of what is believed in a suitably defined set of possible worlds forces safety theorists to make a series of theoretical choices that ultimately prevent a satisfactory solution to the problem. In this way, I analyze several safety principles that result from such choices—the paper thus gives valuable insights into the nature of safety—and explain why none solves the lottery problem, including their inability to explain away Gettierized lottery cases. On a more positive note, I show that there is a viable solution in terms of safety if we get rid of the unquestioned assumption that safe beliefs directly track the truth. The alternative is a conception of safe belief according to which what safe beliefs directly track is the appropriateness of the circumstances and, indirectly, the truth. The resulting safety principle, I argue, explains why mere statistical evidence is not a safe source of knowledge. (shrink)

Many philosophers have argued that statistical evidence regarding group char- acteristics (particularly stereotypical ones) can create normative conflicts between the requirements of epistemic rationality and our moral obligations to each other. In a recent paper, Johnson-King and Babic argue that such conflicts can usually be avoided: what ordinary morality requires, they argue, epistemic rationality permits. In this paper, we show that as data gets large, Johnson-King and Babic’s approach becomes less plausible. More constructively, we build on their project (...) and develop a generalized model of reasoning about stereotypes under which one can indeed avoid normative conflicts, even in a big data world, when data contain some noise. In doing so, we also articulate a general approach to rational belief updating for noisy data. (shrink)

There are many objections to statistical discrimination in general and racial profiling in particular. One objection appeals to the idea that people have a right to be treated as individuals. Statistical discrimination violates this right because, presumably, it involves treating people simply on the basis of statistical facts about groups to which they belong while ignoring non-statistical evidence about them. While there is something to this objection—there are objectionable ways of treating others that seem aptly (...) described as failing to treat them as individuals—it needs to be articulated carefully. First, most people accept that many forms of statistical discrimination are morally unproblematic, let alone morally justified all things considered. Second, even treating people on the basis of putative non-statistical evidence relies on generalizations. Once we construe treating someone as an individual in a way that respects this fact, it becomes apparent: (1) that statistical discrimination is compatible with treating people as individuals, and (2) that one may fail to treat people as individuals even without engaging in statistical discrimination. Finally, there are situations involving the expression of messages of inclusion where we think it is good, morally speaking, that we are not treated as individuals. (shrink)

In the debate about the legal value of naked statistical evidence, Di Bello argues that (1) the likelihood ratio of such evidence is unknown, (2) the decision-theoretic considerations indicate that a conviction based on such evidence is unacceptable when expected utility maximization is combined with fairness constraints, and (3) the risk of mistaken conviction based on such evidence cannot be evaluated and is potentially too high. We argue that Di Bello’s argument for (1) works in (...) a rather narrow context, and that (1) is not exactly in line with the way expert witnesses are required to formulate their opinions. Consequently, Di Bello’s argument for (2), which assumes (1), does not apply uniformly to all convictions based on naked evidence. Moreover, if Di Bello’s analysis is correct, it applies also to eyewitness testimony, given empirical results about its quality, and so the distinctions drawn by DiBello cut across the distinction between naked statistical evidence and other types of evidence. Finally, if we weaken the rather strong requirement of precise measurability of the risk of mistaken conviction, to the availability of reasonable but imprecise and fallible estimates, many field and empirical studies show that often the risk of mistaken conviction based on naked statistical evidence can be estimated to a similar extent as the risk of mistaken conviction based on any other sort of evidence. (shrink)

While the common procedure of statistical significance testing and its accompanying concept of p-values have long been surrounded by controversy, renewed concern has been triggered by the replication crisis in science. Many blame statistical significance tests themselves, and some regard them as sufficiently damaging to scientific practice as to warrant being abandoned. We take a contrary position, arguing that the central criticisms arise from misunderstanding and misusing the statistical tools, and that in fact the purported remedies themselves (...) risk damaging science. We argue that banning the use of p-value thresholds in interpreting data does not diminish but rather exacerbates data-dredging and biasing selection effects. If an account cannot specify outcomes that will not be allowed to count as evidence for a claim—if all thresholds are abandoned—then there is no test of that claim. The contributions of this paper are: To explain the rival statistical philosophies underlying the ongoing controversy; To elucidate and reinterpret statistical significance tests, and explain how this reinterpretation ameliorates common misuses and misinterpretations; To argue why recent recommendations to replace, abandon, or retire statistical significance undermine a central function of statistics in science: to test whether observed patterns in the data are genuine or due to background variability. (shrink)

The replication crisis has prompted many to call for statistical reform within the psychological sciences. Here we examine issues within Frequentist statistics that may have led to the replication crisis, and we examine the alternative—Bayesian statistics—that many have suggested as a replacement. The Frequentist approach and the Bayesian approach offer radically different perspectives on evidence and inference with the Frequentist approach prioritising error control and the Bayesian approach offering a formal method for quantifying the relative strength of (...) class='Hi'>evidence for hypotheses. We suggest that rather than mere statistical reform, what is needed is a better understanding of the different modes of statistical inference and a better understanding of how statistical inference relates to scientific inference. (shrink)

Like many, I find the idea of relying on patient preference predictors in life-or-death cases ethically troubling. As part of his stimulating discussion, Sharadin1 diagnoses such unease as a worry that using PPPs disrespects patients’ autonomy, by treating their most intimate and significant desires as if they were caused by their demographic traits. I agree entirely with Sharadin’s ‘debunking’ response to this concern: we can use statistical correlations to predict others’ preferences without thereby assuming any causal claim. However, I (...) suspect that, for at least some of us, our unease about PPPs stems from a different kind of ‘autonomy’ concern. In this commentary, then, I will explore this concern, and show how it relates to Sharadin’s work. Very many of our preferences are caused, ultimately, by facts which are outside our control, such as our demographic features. However, I suggest that we can still act autonomously on the basis of such preferences, when they are preferences which we endorse. Imagine, for example, that Jane has grown up in a church-growing environment, which has shaped many of her preferences, …. (shrink)

Many legal disputes turn on scientific, especially statistical, evidence. Traditionally scientists have accepted only that statistical evidence which satisfies a 95 percent (or 99 percent) rule — that is, only evidence which has less than five percent (or one percent) probability of resulting from chance.The rationale for this rule is the reluctance of scientists to accept anything less than the best-supported new knowledge. The rule reflects the internal needs of scientific practice. However, when uncritically adopted (...) as a rule for admitting legal evidence, the seemingly innocuous 95 percent rule distorts the balance of interests historically protected by the legal system. In particular, plaintiffs in toxic tort and employment discrimination suits are effectively held to a heavier burden of proof in showing that their injuries were more probably than not caused by the defendant's actions. The result is that too many victims of toxic torts or employment discrimination cannot win legal redress for their injuries. (shrink)