Reports of quantitative experimental results often distinguish between the statistical uncertainty and the systematic uncertainty that characterize measurement outcomes. This article discusses the practice of estimating systematic uncertainty in high-energy physics. The estimation of systematic uncertainty in HEP should be understood as a minimal form of quantitative robustness analysis. The secure evidence framework is used to explain the epistemic significance of robustness analysis. However, the empirical value of a measurement result depends crucially not only on the resulting systematic uncertainty estimate, (...) but on the learning aims for which that result will be used. Important conceptual and practical questions regarding systematic uncertainty assessment call for further investigation. _1_ Introduction _2_ Systematic Uncertainty: The Very Idea _3_ Systematic Uncertainty in High-Energy Physics _4_ Methodological Debates in High-Energy Physics _5_ The Secure Evidence Framework _6_ Systematic Uncertainty Assessment as Robustness Analysis _7_ Security and Sensitivity _8_ Conclusion Appendix. (shrink)

Many philosophers have claimed that evidence for a theory is better when multiple independent tests yield the same result, i.e., when experimental results are robust. Little has been said about the grounds on which such a claim rests, however. The present essay presents an analysis of the evidential value of robustness that rests on the fallibility of assumptions about the reliability of testing procedures and a distinction between the strength of evidence and the security of an evidence claim. Robustness can (...) enhance the security of an evidence claim either by providing what I call second-order evidence, or by providing back-up evidence for a hypothesis. (shrink)

The Evidence for the Top Quark offers both a historical and philosophical perspective on an important recent discovery in particle physics: evidence for the elementary particle known as the top quark. Drawing on published reports, oral histories, and internal documents from the large collaboration that performed the experiment, Kent Staley explores in detail the controversies and politics that surrounded this major scientific result. At the same time the book seeks to defend an objective theory of scientific evidence based on error (...) probabilities. Such a theory provides an illuminating explication of the points of contention in the debate over the evidence for the top quark. Philosophers wishing to defend the objectivity of the results of scientific research must face unflinchingly the realities of scientific practice, and this book attempts to do precisely that. (shrink)

This paper examines the role of evidential considerations in relation to pragmatic concerns in statements of group belief, focusing on scientific collaborations that are constituted in part by the aim of evaluating the evidence for scientific claims (evidential collaborations). Drawing upon a case study in high energy particle physics, I seek to show how pragmatic factors that enter into the decision to issue a group statement contribute positively to the epistemic functioning of such groups, contrary to the implications of much (...) of the existing discussion of group belief. I conclude by suggesting that applying social epistemological considerations to scientific collaborations could be practically beneficial, but only if an appropriately broad range of epistemic values is considered. (shrink)

Experimenters sometimes insist that it is unwise to examine data before determining how to analyze them, as it creates the potential for biased results. I explore the rationale behind this methodological guideline from the standpoint of an error statistical theory of evidence, and I discuss a method of evaluating evidence in some contexts when this predesignation rule has been violated. I illustrate the problem of potential bias, and the method by which it may be addressed, with an example from the (...) search for the top quark. A point in favor of the error statistical theory is its ability, demonstrated here, to explicate such methodological problems and suggest solutions, within the framework of an objective theory of evidence. (shrink)

I consider the error-statistical account as both a theory of evidence and as a theory of inference. I seek to show how inferences regarding the truth of hypotheses can be upheld by avoiding a certain kind of alternative hypothesis problem. In addition to the testing of assumptions behind the experimental model, I discuss the role of judgments of implausibility. A benefit of my analysis is that it reveals a continuity in the application of error-statistical assessment to low-level empirical hypotheses and (...) highly general theoretical principles. This last point is illustrated with a brief sketch of the issues involved in the parametric framework analysis of tests of physical theories such as General Relativity and of fundamental physical principles such as the Einstein Equivalence Principle. (shrink)

Some accounts of evidence regard it as an objective relationship holding between data and hypotheses, perhaps mediated by a testing procedure. Mayo’s error-statistical theory of evidence is an example of such an approach. Such a view leaves open the question of when an epistemic agent is justified in drawing an inference from such data to a hypothesis. Using Mayo’s account as an illustration, I propose a framework for addressing the justification question via a relativized notion, which I designate security , (...) meant to conceptualize practices aimed at the justification of inferences from evidence. I then show how the notion of security can be put to use by showing how two quite different theoretical approaches to model criticism in statistics can both be viewed as strategies for securing claims about statistical evidence. (shrink)

Amidst long-running debates within the field, high energy physics has adopted a statistical methodology that primarily employs standard frequentist techniques such as significance testing and confidence interval estimation, but incorporates Bayesian methods for limited purposes. The discovery of the Higgs boson has drawn increased attention to the statistical methods employed within HEP. Here I argue that the warrant for the practice in HEP of relying primarily on frequentist methods can best be understood as pragmatic, in the sense that statistical methods (...) are chosen for their suitability for the practical demands of experimental HEP, rather than reflecting a commitment to a particular epistemic framework. In particular, I argue that understanding the statistical methodology of HEP through the perspective of pragmatism clarifies the role of and rationale for significance testing in the search for new phenomena such as the Higgs boson. (shrink)

: For philosophers of science interested in elucidating the social character of science, an important question concerns the manner in which and degree to which the objectivity of scientific knowledge is socially constituted. We address this broad question by focusing specifically on philosophical theories of evidence. To get at the social character of evidence, we take an interdisciplinary approach informed by categories from argumentation studies. We then test these categories by exploring their applicability to a case study from high-energy physics. (...) Our central claim is that normative philosophy of science must move beyond abstract theories of justification, confirmation, or evidence conceived impersonally and incorporate a theoretical perspective that includes dialogical elements, either as adjuncts to impersonal theories of evidence or as intrinsic to the cogency of scientific argumentation. (shrink)

Recently, Rueger and Sharp and Koperski have been concerned to show that certain procedural accounts of model confirmation are compromised by non-linear dynamics. We suggest that the issues raised are better approached by considering whether chaotic data analysis methods allow for reliable inference from data. We provide a framework and an example of this approach.

While epistemic justification is a central concern for both contemporary epistemology and philosophy of science, debates in contemporary epistemology about the nature of epistemic justification have not been discussed extensively by philosophers of science. As a step toward a coherent account of scientific justification that is informed by, and sheds light on, justificatory practices in the sciences, this paper examines one of these debates—the internalist-externalist debate—from the perspective of objective accounts of scientific evidence. In particular, we focus on Deborah Mayo’s (...) error-statistical theory of evidence because it is a paradigmatically objective theory of evidence that is strongly informed by methodological practice. We contend that from the standpoint of such an objective theory of evidence, justification in science has both externalist and internalist characteristics. In reaching this conclusion, however, we find that the terms of the contemporary debate between internalists and externalists have to be redefined to be applicable to scientific contexts. (shrink)

: Peter Galison has recently claimed that twentieth-century microphysics has been pursued by two distinct experimental traditions--the image tradition and the logic tradition--that have only recently merged into a hybrid tradition. According to Galison, the two traditions employ fundamentally different forms of experimental argument, with the logic tradition using statistical arguments, while the image tradition strives for non-statistical demonstrations based on compelling ("golden") single events. I show that discoveries in both traditions have employed the same statistical form of argument, even (...) when basing discovery claims on single, golden events. Where Galison sees an epistemic divide between two communities that can only be bridged by a creole- or pidgin-like "interlanguage," there is in fact a shared commitment to a statistical form of experimental argument. (shrink)

Some prominent accounts of scientific evidence treat evidence as an unrelativized concept. But whether belief in a hypothesis is justified seems relative to the epistemic situation of the believer. The issue becomes yet more complicated in the context of group epistemic agents, for then one confronts the problem of relativizing to an epistemic situation that may include conflicting beliefs. As a step toward resolution of these difficulties, an ideal of justification is here proposed that incorporates both an unrelativized evidence requirement (...) and the requirement of the security of the evidence on which a conclusion from data is based. The latter requirement incorporates the consideration of epistemic modal statements. (shrink)

This paper examines probabilistic versions of the fine-tuning argument for design (FTA), with an emphasis on the interpretation of the probability statements involved in such arguments. Three categories of probability are considered: physical, epistemic, and logical. Of the three possibilities, I argue that only logical probability could possibly support a cogent probabilistic FTA. However, within that framework, the premises of the argument require a level of justification that has not been met, and, it is reasonable to believe, will not be (...) met anytime soon. (shrink)

This book guides readers by gradual steps through the central concepts and debates in the philosophy of science. Using concrete examples from the history of science, Kent W. Staley shows how seemingly abstract philosophical issues are relevant to important aspects of scientific practice. Structured in two parts, the book first tackles the central concepts of the philosophy of science, such as the problem of induction, falsificationism, and underdetermination, and important figures and movements, such as the logical empiricists, Thomas Kuhn, and (...) Paul Feyerabend. The second part turns to contemporary debates in the philosophy of science, such as scientific realism, explanation, the role of values in science, the different views of scientific inference, and probability. This broad yet detailed overview will give readers a strong grounding whilst also providing opportunities for further exploration. It will be of particular interest to students of philosophy, the philosophy of science, and science. (shrink)

It is sometimes held that facts confirm a hypothesis only if they were not used in the construction of that hypothesis. This requirement of "use novelty" introduces a historical aspect into the assessment of evidence claims. I examine a methodological principle invoked by physicists in the experimental search for the top quark that bears a striking resemblance to this view. However, this principle is better understood, both historically and philosophically, in terms of the need to conduct a severe test than (...) in terms of use novelty. Nevertheless, a historical factor remains in the assessment of some evidence claims. (shrink)

: Evidence claims depend on fallible assumptions. This paper discusses inferential robustness as a strategy for justifying evidence claims in spite of this fallibility. I argue that robustness can be understood as a means of establishing the partial security of evidence claims. An evidence claim is secure relative to an epistemic situation if it remains true in all scenarios that are epistemically possible relative to that epistemic situation.

I contrast two modes of error-elimination relevant to evaluating evidence in accounts that emphasize frequentist reliability. The contrast corresponds to that between the use of of a reliable inference procedure and the critical scrutiny of a procedure with regard to its reliability, in light of what is and is not known about the setting in which the procedure is used. I propose a notion of security as a category of evidential assessment for the latter. In statistical settings, robustness theory and (...) misspecification testing exemplify two distinct strategies for securing statistical inferences. (shrink)

It is sometimes held that facts confirm a hypothesis only if they were not used in the construction of that hypothesis. This requirement of "use novelty" introduces a historical aspect into the assessment of evidence claims. I examine a methodological principle invoked by physicists in the experimental search for the top quark that bears a striking resemblance to this view. However, this principle is better understood, both historically and philosophically, in terms of the need to conduct a severe test than (...) in terms of use novelty. Nevertheless, a historical factor remains in the assessment of some evidence claims. (shrink)

: Evidence claims depend on fallible assumptions. Three strategies for making true evidence claims in spite of this fallibility are strengthening the support for those assumptions, weakening conclusions, and using multiple independent tests to produce robust evidence. Reliability itself, understood in frequentist terms, does not explain the usefulness of all three strategies; robustness, in particular, sometimes functions in a way that is not well-characterized in terms of reliability. I argue that, in addition to reliability, the security of evidence claims is (...) of epistemic value, where an evidence claim is secure relative to an epistemic situation if it remains true in all scenarios that are epistemically possible relative to that epistemic situation. (shrink)

Evidence claims depend on fallible assumptions. Three strategies for making true evidence claims in spite of this fallibility are strengthening the support for those assumptions, weakening conclusions, and using multiple independent tests to produce robust evidence. Reliability itself, understood in frequentist terms, does not explain the usefulness of all three strategies; robustness, in particular, sometimes functions in a way that is not well-characterized in terms of reliability. I argue that, in addition to reliability, the security of evidence claims is of (...) epistemic value, where an evidence claim is secure relative to an epistemic situation if it remains true in all scenarios that are epistemically possible relative to that epistemic situation. (shrink)

One might view the literature on laws of nature as dividing into two camps: the “metaphysical” advocates of laws as objective realities beyond any actual regularities, and the “antimetaphysical” skeptics. Hard-liners in both camps will find much to disagree with in Marc Lange’s Natural Laws in Scientific Practice. I mean that as a compliment to Lange’s work.

(Almost) All about error Content Type Journal Article Category Book Review Pages 1-5 DOI 10.1007/s11016-011-9618-1 Authors Kent W. Staley, Department of Philosophy, Saint Louis University, 3800 Lindell Blvd., St. Louis, MO 63108, USA Journal Metascience Online ISSN 1467-9981 Print ISSN 0815-0796.

This paper analyzes Deborah Mayo's error-statistical (ES) account of scientific evidence in order to clarify the kinds of "material postulates" it requires and to explain how those assumptions function. A secondary aim is to explain and illustrate the importance of the security of an inference. After finding that, on the most straightforward reading of the ES account, it does not succeed in its stated aims, two remedies are considered: either relativize evidence claims or introduce stronger assumptions. The choice between these (...) approaches turns on the value attached to two aims of inquiry that are in tension: drawing strong, informative conclusions and reasoning securely. (shrink)

The recent discovery of a Higgs boson prompted increased attention of statisticians and philosophers of science to the statistical methodology of High Energy Physics. Amidst long-standing debates within the field, HEP has adopted a mixed statistical methodology drawing upon both frequentist and Bayesian methods, but with standard frequentist techniques such as significance testing and confidence interval estimation playing a primary role. Physicists within HEP typically deny that their methodological decisions are guided by philosophical convictions, but are instead based on “pragmatic” (...) considerations, thus distancing themselves from what they perhaps perceive as an ongoing pitched ideological battle between frequentists and Bayesians. Here I argue that there is a philosophical orientation to HEP that is neither exclusively frequentist nor Bayesian, but that lies squarely in the tradition of philosophical pragmatism. I further argue that understanding the statistical methodology of HEP through the perspective of pragmatism clarifies the role of and rationale for significance testing in the search for new phenomena such as the Higgs boson. (shrink)