Abstract
Amidst long-running debates within the field, high energy physics (HEP) has adopted a statistical methodology that primarily employs standard frequentist techniques such as significance testing and confidence interval estimation, but incorporates Bayesian methods for limited purposes. The discovery of the Higgs boson has drawn increased attention to the statistical methods employed within HEP. Here I argue that the warrant for the practice in HEP of relying primarily on frequentist methods can best be understood as pragmatic, in the sense that statistical methods are chosen for their suitability for the practical demands of experimental HEP, rather than reflecting a commitment to a particular epistemic framework. In particular, I argue that understanding the statistical methodology of HEP through the perspective of pragmatism clarifies the role of and rationale for significance testing in the search for new phenomena such as the Higgs boson.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
O’Hagan collected and summarized the many replies he received to his post. In this digest, he noted that he had intentionally used somewhat inflammatory language to “provoke discussion” (O’Hagan 2012).
For a well-informed guided tour of those paths, with some novel insights, see Sprenger (2016) who cites the Higgs case as motivation for a careful consideration of the issues.
When confronted with a statistical model with multiple parameters, all but one of which are considered ‘nuisance’ parameters, the profile likelihood for the parameter of interest is obtained by maximizing, for every considered value of the parameter of interest, the likelihoods for the each of the nuisance parameters, and then using the values for the nuisance parameters thus obtained for estimating the parameter of interest (Cox 1970; Venzon and Moolgavkar 1988).
As Cousins states, “for each mass [\(m_H\)] there is a p value for the departure from \(H_0\), as if that mass had been fixed in advance” (Cousins 2014, p. 33, emphasis in original).
I have adopted this felicitous expression from a comment by a referee.
The principle invoked here is similar to what Mayo and Spanos call the “weak severity principle” (Mayo and Spanos 2009, p. 21).
I am grateful to an anonymous referee for pressing this issue.
Allan Franklin has documented the emergence of the \(5\sigma \) standard in HEP (Franklin 2013). According to Franklin’s narrative, the standard has only assumed the weight that it does carry rather recently, around the time of the discovery of the top quark, for which an initial paper by CDF in 1994 claimed only “evidence,” with a significance corresponding to \(2.8\sigma \) for a Gaussian distribution (Abe et al. 1994). Later papers by CDF and D0 claimed the top’s “observation” on the basis of \(5.0\sigma \) and \(4.6\sigma \), respectively (Abe et al. 1995; Abachi et al. 1995; Staley 2004).
As noted above, however, Dawid (2015a) uses Bayesian considerations to argue that the LEE is not as significant a problem for the Higgs search as others have suggested, and therefore application of the stringent \(5\sigma \) standard is unjustified.
Those familiar with the “argument from inductive risk” that seeks to establish a role for value judgments in core tasks of scientific reasoning (Churchman 1948; Douglas 2009; Rudner 1953) will note its resemblance to the point being here pursued. See Staley (2016) for further discussion of inductive risk in the context of the Higgs search.
Exactly which framework is appropriate for which problem remains a matter of dispute. Statistical Methods in Experimental Physics, a widely used text, emphasizes (in both of its two editions) frequentist techniques for the analysis of data, and introduces Bayesian statistics as an approach to decision problems (alongside frequentist methods) (Eadie et al. 1971; James 2006).
Anderson also discusses the possibility that an undetected photon struck a nucleus in the lead plate, knocking out two particles in opposite directions. This, however, does not really count as an alternative to the claim of a positively charged electron, since (from considerations of curvature and direction) one of the particles knocked out of the lead plate would have to be just such a positively charged, low-mass particle.
I am not denying that CMS and ATLAS gave detailed presentations; on the contrary, they accompanied a careful evidential argument with very thorough presentations of the analyses they applied to the data. What I am claiming is that conventions of scientific communication tend to suppress explicit discussion of those considerations of the consequences of accepting a claim that play a role in determining the evidential standard of acceptance.
References
Aad, G., Abajyan, T., Abbott, B., Abdallah, J., Abdel Khalek, S., Abdelalim, A. A., et al. (2012a). Combined search for the standard model Higgs boson in pp collisions at \(\sqrt{s}\) = 7 TeV with the ATLAS detector. Physical Review D, 86, 032003. doi:10.1103/PhysRevD.86.032003.
Aad, G., Abajyan, T., Abbott, B., Abdallah, J., Abdel Khalek, S., Abdelalim, A. A., et al. (2012b). Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC. Physics Letters B, 716, 1–29. doi:10.1016/j.physletb.2012.08.020.
Abachi, S., Abbott, B., Abolins, M., Acharya, B. S., Adam, I., Adams, D. L., et al. (1995). Observation of the top quark. Physical Review Letters, 74, 2632–2637. doi:10.1103/PhysRevLett.74.2632.
Abe, F., Akimoto, H., Akopian, A., Albrow, M. G., Amendolia, S. R., Amidei, D., et al. (1995). Observation of top quark production in \(\bar{p}p\) collisions with the collider detector at Fermilab. Physical Review Letters, 74, 2626–2631. doi:10.1103/PhysRevLett.74.2626.
Abe, F., Albrow, M. G., Amendolia, S. R., Amidei, D., Antos, J., Anway- Wiese, C., et al. (1994). Evidence for top quark production in \(\bar{p}p\) collisions at \(\sqrt{s}\) = 1.8 TeV. Physical Review D, 50, 2966–3026. doi:10.1103/PhysRevD.50.2966.
Achinstein, P. (2013). Evidence and method: Scientific strategies of Isaac Newton and James Clerk Maxwell. New York: Oxford University Press.
Anderson, C. (1933). The positive electron. Physical Review, 43, 491–494.
ATLAS. (2012). Latest results from ATLAS Higgs search (Press Release).
Chatrchyan, S., Khachatryan, V., Sirunyan, A., Tumasyan, A., Adam, W., Bergauer, T., et al. (2012a). Combined results of searches for the standard model Higgs boson in pp collisions at \(\sqrt{s}\) = 7 TeV. Physics Letters B, 710(1), 26–48.
Chatrchyan, S., Khachatryan, V., Sirunyan, A., Tumasyan, A., Adam, W., Bergauer, T., et al. (2012b). Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC. Physics Letters B, 716, 30–61. doi:10.1016/j.physletb.2012.08.021.
Churchman, C. W. (1948). Statistics, pragmatics, induction. Philosophy of Science, 15(3), 249–268.
CMS. (2012). Observation of a new particle with a mass of 125 GeV (Press Release).
Cousins, R. D. (2014). The Jeffreys–Lindley paradox and discovery criteria in high energy physics. Synthese, 1–38. doi:10.1007/s11229-014-0525-z.
Cousins, R. D., & Highland, V. L. (1992). Incorporating systematic uncertainties into an upper limit. Nuclear Instruments and Methods in Physics Research, A320, 331–335.
Cox, D. R. (1970). Analysis of binary data. London: Methuen.
Dawid, R. (2015a). Bayesian perspectives on the discovery of the higgs particle. Synthese, 1–18. doi:10.1007/s11229-015-0943-6
Dawid, R. (2015b). Higgs discovery and the look elsewhere effect. Philosophy of Science, 82(1), 76–96.
Douglas, H. (2009). Science, policy, and the value-free ideal. Pittsburgh, PA: University of Pittsburgh Press.
Eadie, W. T., Dryard, D., James, F. E., Roos, M., & Sadoulet, B. (1971). Statistical methods in experimental physics. Amsterdam: North Holland.
Feldman, G. J., & Cousins, R. D. (1998). Unified approach to the classical statistical analysis of small signals. Physical Review D, 57, 3873–3889. doi:10.1103/PhysRevD.57.3873.
Fisher, R. A. (1925). Statistical methods for research workers. Edinburgh: Oliver and Boyd.
Franklin, A. (2013). Shifting standards: Experiments in particle physics in the twentieth century. Pittsburgh, PA: University of Pittsburgh Press.
Harlander, R. V., & Kilgore, W. B. (2002). Next-to-next-to-leading order Higgs production at hadron colliders. Physical Review Letters, 88, 201801. doi:10.1103/PhysRevLett.88.201801.
James, F. (2006). Statistical methods in experimental physics (2nd ed.). Singapore: World Scientific.
Levi, I. (1962). On the seriousness of mistakes. Philosophy of Science, 29(1), 47–65.
Lyons, L. (2013). Discovering the significance of \(5\sigma \). arXiv:1310.1284.
Massimi, M., & Bhimji, W. (2015). Computer simulations and experiments: The case of the Higgs boson. Studies in History and Philosophy of Modern Physics, 51, 71–81.
Mayo, D. G., & Spanos, A. (Eds.). (2009). Error and inference: Recent exchanges on experimental reasoning, reliability, objectivity, and rationality. New York: Cambridge University Press.
Morrison, M. (2015). Reconstructing reality: Models, mathematics, and simulations. New York: Oxford University Press.
O’Hagan, T. (2012). Higgs boson digest and discussion. Retrieved March 17, 2014, from http://bayesian.org/forums/news/3830.
Overbye, D. (2012, July 4). Physicists find elusive particle seen as key to universe. New York Times.
Peirce, C. S. (1883). A theory of probable inference. In C. S. Peirce (Ed.), Studies in logic: By members of the Johns Hopkins University (pp. 126–181). Boston: Little, Brown, and Company.
Rehg, W., & Staley, K. W. (2008). The CDF collaboration and argumentation theory: The role of process in objective knowledge. Perspectives on Science, 16, 1–25.
Rudner, R. (1953). The scientist qua scientist makes value judgments. Philosophy of Science, 20(1), 1–6.
Sprenger, J. (2016). Bayesianism vs frequentism in statistical inference. In A. Hajek & C. Hitchcock (Eds.), The Oxford handbook of probability and philosophy. Oxford: Oxford University Press.
Staley, K. W. (2002). What experiment did we just do? Counterfactual error statistics and uncertainties about the reference class. Philosophy of Science, 69(2), 279–299.
Staley, K. W. (2004). The evidence for the top quark: Objectivity and bias in collaborative experimentation. New York: Cambridge University Press.
Staley, K. W. (2016). Decisions, decisions: Inductive risk and the higgs boson. In K. C. Elliott & T. Richards (Eds.), Exploring inductive risk. New York: Oxford University Press.
Venzon, D. J., & Moolgavkar, S. H. (1988). A method for computing profile-likelihood-based confidence intervals. Journal of the Royal Statistical Society: Series C (Applied Statistics), 37(1), 87–94.
Wickham, C., & Evans, R. (2012, July 4). “It’s a boson”: Higgs quest bears new particle. Reuters.
Willing, R. (2013). Measurement uncertainty and probability. New York: Cambridge University Press.
Acknowledgments
I would like to thank Richard Dawid for his editorial patience and encouragement. This paper grew out of a presentation at a conference organized by Michael Stöltzner, and I would like to thank Michael and other participants, including Richard Dawid, Hugo Beauchemin, Robert Cousins, and Koray Karaca for their insights into these issues. Conversations with Deborah Mayo and Allan Franklin were also helpful. I am especially grateful to three anonymous referees whose generous comments on earlier drafts were very helpful in improving this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Staley, K.W. Pragmatic warrant for frequentist statistical practice: the case of high energy physics. Synthese 194, 355–376 (2017). https://doi.org/10.1007/s11229-016-1111-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11229-016-1111-3