Hostname: page-component-76fb5796d-wq484 Total loading time: 0 Render date: 2024-04-27T20:16:04.027Z Has data issue: false hasContentIssue false
  • English
  • Français

Inherent Complexity: A Problem for Statistical Model Evaluation

Published online by Cambridge University Press:  01 January 2022

Jan-Willem Romeijn
Get access Rights & Permissions [Opens in a new window]

Abstract

This article investigates a problem for statistical model evaluation, in particular for curve fitting: by employing a different family of curves we can fit any scatter plot almost perfectly at apparently minor cost in terms of model complexity. The problem is resolved by an appeal to prior probabilities. This leads to some general lessons about how to approach model evaluation.

Type
Evidence and Inference
Information
Philosophy of Science , Volume 84 , Issue 5 , December 2017 , pp. 797 - 809
Copyright
Copyright © The Philosophy of Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

†

The author wishes to thank Aidan Lyon, Elliott Sober, Jan Sprenger, and Tom Sterkenburg, as well as audiences in Santiago de Compostella, Padova, Helsinki, Gent, and Groningen.

References

Akaike, H. 1973. “Information Theory and an Extension of the Maximum Likelihood Principle.” In 2nd International Symposium on Information Theory, ed. Petrov, B. N. and Csaki, F., 267–81. Budapest: Akademiai Kiado.Google Scholar
Balasubramanian, V. 2005. “MDL, Bayesian Inference, and the Geometry of the Space of Probability Distributions.” In Advances in Minimum Description Length: Theory and Applications, ed. Grunwald, P. J. et al., 81–99. Cambridge, MA: MIT Press.Google Scholar
Bozdogan, H. 1987. “Model Selection and Akaike’s Information Criterion.” Psychometrika 52 (3): 345–70.CrossRefGoogle Scholar
Burnham, K. P., and Anderson, D. R. 2002. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. New York: Springer.Google Scholar
Claeskens, G., and Hjort, N. L. 2008. Model Selection and Model Averaging. Cambridge: Cambridge University Press.Google Scholar
Grunwald, P. 2007. The Minimum Description Length Principle. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
Harman, G., and Kulkarni, S. 2007. Reliable Reasoning. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
Ly, A., Verhagen, J., Grasman, R., and Wagenmakers, E.-J. 2017. “A Tutorial on Fisher Information.” Unpublished manuscript, University of Amsterdam.CrossRefGoogle Scholar
Myung, J., et al. 2000. “Counting Probability Distributions: Differential Geometry and Model Selection.” Proceedings of the National Academy of Sciences 97 (21): 11170–75.CrossRefGoogle ScholarPubMed
Raftery, A. E. 1995. “Bayesian Model Selection in Social Research.” Sociological Methodology 25:111–63.Google Scholar
Romeijn, J. W., and van de Schoot, R. 2008. “A Philosophical Analysis of Bayesian Model Selection.” In Bayesian Evaluation of Informative Hypotheses, ed. Hoijtink, H., Klugkist, I., and Boelen, P. A. New York: Springer.Google Scholar
Romeijn, J. W., van de Schoot, R., and Hoijtink, H. 2012. “One Size Does Not Fit All: Derivation of a Prior-Adapted BIC.” In Probabilities, Laws, and Structures, ed. Dieks, D., Gonzales, W., Hartmann, S., Stadler, F., Uebel, T., and Weber, M. Berlin: Springer.Google Scholar
Schwarz, G. 1978. “Estimating the Dimension of a Model.” Annals of Statistics 6:461–64.CrossRefGoogle Scholar
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., and van der Linde, A. 2002. “Bayesian Measures of Model Complexity and Fit.” Journal of Royal Statistical Society B 64:583–639.CrossRefGoogle Scholar