Skip to main content
Log in

How simulations fail

Synthese Aims and scope Submit manuscript

Abstract

‘The problem with simulations is that they are doomed to succeed.’ So runs a common criticism of simulations—that they can be used to ‘prove’ anything and are thus of little or no scientific value. While this particular objection represents a minority view, especially among those who work with simulations in a scientific context, it raises a difficult question: what standards should we use to differentiate a simulation that fails from one that succeeds? In this paper we build on a structural analysis of simulation developed in previous work to provide an evaluative account of the variety of ways in which simulations do fail. We expand the structural analysis in terms of the relationship between a simulation and its real-world target emphasizing the important role of aspects intended to correspond and also those specifically intended not to correspond to reality. The result is an outline both of the ways in which simulations can fail and the scientific importance of those various forms of failure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  • Axelrod R., Hamilton W. (1981) The evolution of cooperation. Science 211: 1390–1396

    Article  Google Scholar 

  • Axtell, R. L., Epstein, J. M., Dean, J. S., Gumerman, G. J., Swedland, A. C., Harburger, J., Chakravarty, S., Hammond, R., Parker, J., & Parker, M. (2002). Population growth and collapse in a multiagent model of the Kayenta Anasazi in Long House Valley. In B. J. L.Berry, L. D. Kiel, & E. Eliott (Eds.), Adaptive agents, intelligence, and emergent human organization: Capturing complexity through agent-based modeling Proceedings of the National Academy of Sciences of the USA, (Vol. 99, Suppl. 3, pp. 7275–7279). Washington, DC: National Academy of Sciences.

  • Barberousse A., Franceschelli S., Imbert C. (2009) Computer simulations as expriments. Synthese 169: 557–574

    Article  Google Scholar 

  • Borges, J. L. (1998). Collected fictions (A. Hurley, Trans.). New York, NY: Penguin Books.

  • Bottke W., Vokrouhlický D., Nesvorný D. (2007) An asteroid breakup 160 myr ago as the probable source of the K/T impactor. Nature 449: 48–53

    Article  Google Scholar 

  • Bruch, E. E., & Mare, R. D. (2001). Spatial inequality, neighborhood mobility, and residential segregation. Los Angeles, CA: California Center for Population Research On-Line Working Paper Series.

  • Bruch E. E., Mare R. D. (2006) Neighborhood choice and neighborhood change. American Journal of Sociology 112: 667–709

    Article  Google Scholar 

  • Canup R. M. (2004) Simulations of a late lunar forming impact. Icarus 168: 433–456

    Article  Google Scholar 

  • Cartwright N. (1983) How the laws of physics lie. Oxford University Press, Oxford

    Book  Google Scholar 

  • Cartwright, N. (1999). Aristotelian natures and modern experimental method. In The dappled world. Cambridge: Cambridge University Press.

  • Chattoe E., Saam N. J., Möhring M. (2000) Sensitivity analysis in the social sciences: Problems and prospects. In: Suleiman R., Troitsch K. G., Gilbert N. (eds) Tools and techniques for social science simulation. Physica-Verlag, pp 243–273 Heidelberg

    Google Scholar 

  • Committee on Modeling Community Containment for Pandemic Influenza. (2006). Modeling community containment for pandemic influenza: A letter report. Institute of Medicine of the National Academies. http://www.nap.edu/catalog/11800.html

  • Cummings D. A. T., Chakravarty S., Singha R. M., Burke D. S., Epstein J. M. (2004) Toward a containment strategy for smallpox bioterror: An individual-based computational approach. Brookings Institute Press, Washington, DC

    Google Scholar 

  • Da Costa N., French S. (2003) Science and partial truth: A unitary approach to models and scientific reasoning. Oxford University Press, Oxford

    Google Scholar 

  • Dean J. S., Gumerman G. J., Epstein J., Axtell R. L., Swedland A. C., Parker M. T., McCarrol S. (1999) Understanding Anasazi culture change through agent based modeling. In: Kohler T. A., Gumerman G. J. (eds) Dynamics in human and primate societies: Agent based modeling of social and spatial processes. Oxford University Press, New York, NY, pp 179–206

    Google Scholar 

  • Eason R., Rosenberger R., Kokalis T., Selinger E., Grim P. (2007) What kind of science is simulation?. Journal of Experimental and Theoretical Artificial Intelligence 19(1): 19–28

    Article  Google Scholar 

  • Elgin C. (2009) Exemplification, idealization, and scientific understanding. In: Suárez M. (eds) Fictions in science: Philosophical essays on modeling and idealization. Routledge, New York, NY

    Google Scholar 

  • Epstein J. M. (2002) Modeling civil violence: An agent-based computational approach. Proceedings of the National Academy of Sciences, USA 99: 7243–7250

    Article  Google Scholar 

  • Ferguson N. M., Cummings D. A. T., Fraser C., Cajka J. C., Cooley P. C., Burke D. S. (2006) Strategies for mitigating an influenza pandemic. Nature 442: 448–451

    Article  Google Scholar 

  • Frigg, R., & Hartmann, S. (2006). Models in science. Stanford encyclopedia of philosophy. http://plato.stanford.edu/entries/models-science.

  • Giere R. (2004) How models are used to represent reality. Philosophy of Science 71(Supplement): S742–752

    Article  Google Scholar 

  • Glymour C., Scheines R., Spirtes P., Kelly K. (1987) Discovering causal structure: Artificial intelligence, philosophy of science, and statistical modeling. Academic Press, San Diego, CA

    Google Scholar 

  • Granovetter M. (1978) Threshold models of collective behavior. American Sociological Review 83: 1420–1442

    Article  Google Scholar 

  • Grim P. (1995) Greater generosity in the spatialized Prisoner’s Dilemma. Journal of Theoretical Biology 173: 353–359

    Article  Google Scholar 

  • Grim P. (1996) Spatialization and greater generosity in the stochastic Prisoner’s Dilemma. Biosystems 37: 3–17

    Article  Google Scholar 

  • Grim P., Mar G., St. Denis P. (1998) The philosophical computer: Exploratory essays in philosophical computer modeling. MIT Press, Cambridge, MA

    Google Scholar 

  • Grim P., Au R., Louie N., Rosenberger R., Braynen W., Selinger E., Eason R. E (2006) Game-theoretic robustness in cooperation and prejudice reduction: A graphic measure. In: Rocha L. M., Yaeger L. S., Bedau M. A., Floreano D., Goldstone R. L., Vespignani A. (eds) Artificial life X. MA: MIT Press, Cambridge, pp 445–451

    Google Scholar 

  • Grim P., Au R., Louie N., Rosenberger R., Braynen W., Selinger E., Eason R. E. (2008) A graphic measure for game theoretic robustness. Synthese 163(2): 273–297

    Article  Google Scholar 

  • Grim P., Selinger E., Braynen W., Rosenberger R., Au R., Louie N., Connolly J. (2004) Reducing prejudice: A spatialized game-theoretic model for the contact hypothesis. In: Pollack J., Bedau M., Husbands P., Ikegami T., Watson R. A. (eds) Artificial life IX. MA: MIT Press, Cambridge, pp 244–249

    Google Scholar 

  • Grim P., Selinger E., Braynen W., Rosenberger R., Au R., Louie N., Connolly J. (2005) Modeling prejudice reduction: Spatialized game theory and the contact hypothesis. Public Affairs Quarterly 19: 95–125

    Google Scholar 

  • Guala F. (2002) Models, simulations, and experiments. In: Magnani L., Nersessian N. (eds) Model-based reasoning: Science, technology, values. Kluwer, New York, NY, pp 59–74

    Chapter  Google Scholar 

  • Gumerman G. J., Swedland A. C., Dean J. S., Epstein J. M. (2003) The evolution of social behavior in the prehistoric American Southwest. Artificial Life 9: 435–444

    Article  Google Scholar 

  • Huberman B., Glance N. (1993) Evolutionary games and computer simulations. Proceedings of the National Academy of Science, USA 90: 7716–7718

    Article  Google Scholar 

  • Huggins E. M., Schultz E. A. (1967) San Francisco bay in a warehouse. Journal of the Institute of Environmental Sciences and Technology 10(5): 9–16

    Google Scholar 

  • Huggins, E. M., & Schultz, E. A. (1973). The San Francisco bay and delta model. California Engineer, 51(3), 11–23. http://www.spn.usace.army.mil/bmvc/bmjourney/the_model/history.html.

  • Interagency Performance Evaluation Task Force. (2006). Performance evaluation of the New Orleans and Southeast Louisiana Hurricane Protection System: Draft final report of the Interagency Performance Evaluation Task Force (Vol. 1). www.asce.org/files/pdf/executivesummary_v20i.pdf.

  • Kitcher P. (1993) The evolution of human altruism. Journal of Philosophy 90: 497–516

    Article  Google Scholar 

  • Küppers G., Lenhard J. (2006) From hierarchical to network-like integration: A revolution of modeling style in computer-simulation. In: Küppers G., Lenhard J., Shinn T. (eds) Simulation: Pragmatic construction of reality. Springer, Dordrecht, pp 89–106

    Google Scholar 

  • Mayo, D., Hollander, R. D. (eds) (1994) Acceptable evidence: Science and values in risk management. University Press, New York, NY

    Google Scholar 

  • Morgan M. (2003) Experiments without material intervention: Model experiments, virtual experiments and virtually experiments. In: Radder H. (eds) The philosophy of scientific experimentation. University of Pittsburgh Press, Pittsburgh, pp 216–235

    Google Scholar 

  • Ngenkaew, W., Ono, S., & Nakayama, S. (2007). Multiple pheromone deposition in ant-based clustering as an ant foraging concept. In S. Sahni (Ed.), Proceedings of the 3rd IASTED International Conference, Advances in computer science and technology (pp. 432–436). Anaheim, CA: Acta Press.)

  • Nowak M., May R. (1993) The spatial dimensions of evolution. International Journal of Bifurcation and Chaos 3: 35–78

    Article  Google Scholar 

  • Nowak M., Sigmund K. (1992) Tit For Tat in heterogeneous populations. Nature 355: 250–252

    Article  Google Scholar 

  • Parker W. S. (2008) Computer simulation through an error-statistical lens. Synthese 163(3): 371–384

    Article  Google Scholar 

  • Parker W. S. (2009) Does matter really matter? Computer simulations, experiments, and reality. Synthese 169: 483–496

    Article  Google Scholar 

  • Pearl J. (2000) Causality: Models, reasoning, and inference. Cambridge University Press, Cambridge, MA

    Google Scholar 

  • Rescher N. (1998) Predicting the future. SUNY Press, Albany, NY

    Google Scholar 

  • Resnick M. (1997) Turtles, termites, and traffic jams: Explorations in massively parallel microworlds. MIT Press, Cambridge, MA

    Google Scholar 

  • Robinson, M. C. (1992). Rivers in miniature: The Mississippi Basin Model. In B. W. Fowle (Ed.), Builders and fighters: U.S. Army Engineers in World War II. Fort Belvoir, VA: Office of History, United States Army Corps of Engineers.

  • Schelling T. C. (1978) Micromotives and macrobehavior. Norton, New York, NY

    Google Scholar 

  • Smith J. M. (1995) Life at the edge of chaos?. New York Review of Books 42(4): 28–30

    Google Scholar 

  • Spirtes P., Glymour C., Scheines R. (2000) Causation, prediction, and search (2nd ed.). MIT Press, Cambridge, MA

    Google Scholar 

  • Sterrett S. G. (2005) Wittgenstein flies a kite. Pi Press, New York, NY

    Google Scholar 

  • Suarez M. (2003) Scientific representation: Against similarity and isomorphism. International Studies in the Philosophy of Science 7: 225–244

    Article  Google Scholar 

  • Suarez, M. (eds) (2009) Fictions in science: Philosophical essays on modeling and idealization. Routledge, New York, NY

    Google Scholar 

  • Suppes P. (2002) Representation and invariance of scientific structures. CSLI Publications, Stanford, CA

    Google Scholar 

  • Teller P. (2001) Twilight of the perfect model. Erkenntnis 55: 393–415

    Article  Google Scholar 

  • van Fraassen B. C. (1980) The scientific image. Oxford University Press, Oxford

    Book  Google Scholar 

  • van Fraassen B. C. (2008) Scientific representation: Paradoxes of perspective. Oxford University Press, New York, NY

    Book  Google Scholar 

  • Walton K. (1990) Mimesis as make-believe. Harvard University Press, Cambridge, MA

    Google Scholar 

  • Winsberg E. (2003) Simulated experiments: Methodology for a virtual world. Philosophy of Science 70: 105–125

    Article  Google Scholar 

  • Winsberg E. (2009) A tale of two methods. Synthese 169: 575–592

    Article  Google Scholar 

  • Woodward J. B. (2003) Making things happen: A theory of causal explanation. Oxford University Press, New York, NY

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Robert Rosenberger.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Grim, P., Rosenberger, R., Rosenfeld, A. et al. How simulations fail. Synthese 190, 2367–2390 (2013). https://doi.org/10.1007/s11229-011-9976-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11229-011-9976-7

Keywords

Navigation