Skip to main content
SpringerLink
Log in

Challenges to Simulation Validation in the Social Sciences. A Critical Rationalist Perspective

  • Chapter
  • First Online:
Computer Simulation Validation

Part of the book series: Simulation Foundations, Methods and Applications ((SFMA))

Abstract

I reflect on challenges to the validation of theoretical models from the perspective of a critical rationalist seeking to develop true explanations of empirical phenomena. I illustrate my arguments with examples from the rich literature on social-influence models, a field that has profited from contributions from various disciplines such as physics, and mathematics. While this field is characterized by a large number of competing formal models, it has been criticized for having failed to generate reliable explanations and predictions, because of a lack of empirical research validating models. I list five challenges to model validation in the social sciences: First, social-scientific theories are based on many obscure concepts. Second, many social-scientific concepts are latent. Third, the representation of time is unclear in many models. Forth, in most social settings, various processes influence dynamics in parallel. Fifth, context dependencies limit the development of general models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The concept of validity has been used in many different ways. I use it here in accordance with many contributions to the social-simulation literature (David 2009), defining a model as valid if it is based on true assumptions. In the field of logic, one would call such a theory “sound” rather than “valid,” because in this literature explanations are considered valid when their assumptions logically imply the explanandum.

  2. 2.

    To be sure, the statement “I feel that I hate rain” can be true (if I do hate rain) or false (If I do not hate rain). Nevertheless, an individual’s evaluation of rain as being negative cannot be described with the words “true” or “false”.

  3. 3.

    https://gssdataexplorer.norc.org/variables/4971/vshow.

References

  • Abelson, R. P. (1964). Mathematical models of the distribution of attitudes under controversy. In N. Frederiksen & H. Gulliksen (Eds.), Contributions to mathematical psychology (pp. 142–160). New York: Rinehart Winston.

    Google Scholar 

  • Ahrweiler, P., & Gilbert, N. (2015). The quality of social simulation: An example from research policy modelling. In M. Janssen, M. Wimmer, & A. Deljoo (Eds.), Policy practice and digital science (pp. 35–55). Heidelberg/New York: Springer.

    Chapter  Google Scholar 

  • Axelrod, R. (1997). The dissemination of culture-A model with local convergence and global polarization. Journal of Conflict Resolution, 41(2), 203–226.

    Article  Google Scholar 

  • Baldassarri, D., & Bearman, P. (2007). Dynamics of political polarization. American Sociological Review, 72(5), 784–811.

    Article  Google Scholar 

  • Bikhchandani, S., Hirshleifer, D., & Welch, I. (1992). A theory of fads, fashion, custom, and cultural-change as informational cascades. Journal of Political Economy, 100(5), 992–1026.

    Article  Google Scholar 

  • Bonacich, P., & Philip, L. (2012). Introduction to mathematical sociology. Princeton and Oxford: Princeton University Press.

    MATH  Google Scholar 

  • Boudon, R. (1981). The logic of social action. An introduction to sociological analysis. London: Routledge and Kegan Paul.

    Google Scholar 

  • Bourdieu, P. (1984). Distinction: A social critique of the judgment of taste. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Boxell, L., Gentzkow, M., & Shapiro, J. M. (2017). Greater internet use is not associated with faster growth in political polarization among US demographic groups. Proceedings of the National Academy of Sciences of the United States of America, 114(40), 10612–10617.

    Article  Google Scholar 

  • Bramson, A., Grim, P., Singer, D. J., Fisher, S., Berger, W., Sack, G., & Flocken, C. (2016). Disambiguation of social polarization concepts and measures. The Journal of Mathematical Sociology.

    Google Scholar 

  • Brewer, M. B. (1991). The social self-On being the same and different at the same time. Personality and Social Psychology Bulletin, 17(5), 475–482.

    Article  Google Scholar 

  • Brousmiche, K.-L., Kant, J.-D., Sabouret, N., & Prenot-Guinard, F. (2016). From beliefs to attitudes: Polias, a model of attitude dynamics based on cognitive modeling and field data. Journal of Artificial Societies and Social Simulation, 19(4).

    Google Scholar 

  • Byrne, D. (1971). The attraction paradigm. New York, London: Academic Press.

    Google Scholar 

  • Castellano, C., Fortunato, S., & Loreto, V. (2009). Statistical physics of social dynamics. Reviews of Modern Physics, 81(2), 591–646.

    Article  Google Scholar 

  • Chattoe-Brown, E. (2014). Using agent based modelling to integrate data on attitude change. Sociological Research Online, 19(1).

    Article  Google Scholar 

  • Clemm von Hohenberg, B., Mäs, M., & Pradelski, B. S. R. (2017). Micro influence and macro dynamics of opinion formation. SSRN. https://ssrn.com/abstract=2974413.

  • Conte, R., Gilbert, N., Bonelli, G., Cioffi-Revilla, C., Deffuant, G., Kertesz, J., et al. (2012). Manifesto of computational social science. The European Physical Journal Special Topics, 214, 325–346.

    Article  Google Scholar 

  • David, N. (2009). Validation and verification in social simulation: Patterns and clarification of terminology. In F. Squazzoni (Ed.), EPOS 2006: Epistemological Aspects of Computer Simulation in the Social Sciences (pp. 117–129). Berlin: Springer.

    Chapter  Google Scholar 

  • Deffuant, G., Huet, S., & Amblard, F. (2005). An individual-based model of innovation diffusion mixing social value and individual benefit. American Journal of Sociology, 110(4), 1041–1069.

    Article  Google Scholar 

  • DiMaggio, P., Evans, J., & Bryson, B. (1996). Have Americans’ social attitudes become more polarized? American Journal of Sociology, 102(3), 690–755.

    Article  Google Scholar 

  • Elias, N. (1969). The civilizing process. In The history of manners (Vol. I). Oxford: Blackwell.

    Google Scholar 

  • Esteban, J. M., & Ray, D. (1994). On the measurement of polarization. Econometrica, 62(4), 819–851.

    Article  MATH  Google Scholar 

  • Everett, J. A. C. (2013). The 12 item social and economic conservatism Scale (SECS). PLoS One, 8(12), e82131. (Edited by Pete Roma. Public Library of Science).

    Google Scholar 

  • Flache, A., & Mäs, M. (2008a). Why do faultlines matter? A computational model of how strong demographic faultlines undermine team cohesion. Simulation Modelling Practice and Theory, 16(2), 175–191.

    Article  Google Scholar 

  • Flache, A., & Mäs, M. (2008b). How to get the timing right. A computational model of the effects of the timing of contacts on team cohesion in demographically diverse teams. Computational and Mathematical Organization Theory, 14(1), 23–51.

    Article  MATH  Google Scholar 

  • Flache, A., & Macy, M. W. (2011a). Small worlds and cultural polarization. The Journal of Mathematical Sociology, 35(1–3), 146–176.

    Article  MathSciNet  Google Scholar 

  • Flache, A., & Macy, M. W. (2011b). Local convergence and global diversity: From interpersonal to social influence. Journal of Conflict Resolution, 55(6), 970–995.

    Article  Google Scholar 

  • Flache, A., Mäs, M., Feliciani, T., Chattoe-Brown, E., Deffuant, G., Huet, S., & Lorenz, J. (2017). Models of social influence: Towards the next frontiers. Journal of Artificial Societies and Social Simulation, 20(4).

    Google Scholar 

  • Friedkin, N. E., & Johnsen, E. C. (2011). Social influence network theory. New York: Cambridge University Press.

    Book  MATH  Google Scholar 

  • Garrison, L. A., & Babcock, D. S. (2009). Alcohol consumption among college students: An agent-based computational simulation. Complexity, 14(6), 35–44 (Wiley Subscription Services, Inc., A Wiley Company).

    Google Scholar 

  • Gilbert, N., & Troitzsch, K. G. (1999). Simulation for the social scientist. Buckingham Philadelphia: Open University Press.

    Google Scholar 

  • Gilbert, N., & Ahrweiler, P. (2005). Caffè Nero: The evaluation of social simulation. Journal of Artificial Societies and Social Simulation, 8(4).

    Google Scholar 

  • Golder, S. A., & Macy, M. W. (2014). Digital footprints: Opportunities and challenges for online social research. Annual Review of Sociology, 40(1), 129–152.

    Article  Google Scholar 

  • Grow, A., & Flache, A. (2011). How attitude certainty tempers the effects of faultlines in demographically diverse teams. Computational and Mathematical Organization Theory, 17(2), 196–224.

    Article  MATH  Google Scholar 

  • Hedström, P., & Udehn, L. (2009). Analytical sociology and theories of the middle range. In P. Hedström, & P. Bearman (Eds.), The oxford handbook (pp. 25–47). Oxford University Press.

    Google Scholar 

  • Hedström, P., & Ylikoski, P. (2010). Causal mechanisms in the social sciences. Annual Review of Sociology, 36(1), 49–67.

    Article  Google Scholar 

  • Hegselmann, R., & Krause, U. (2002). Opinion dynamics and bounded confidence models, analysis, and simulation. Journal of Artificial Societies and Social Simulation, 5(3).

    Google Scholar 

  • Hegselmann, R., Schelling, T. C., & Sakoda, J. M. (2017). The intellectual, technical, and social history of a model. Journal of Artificial Societies and Social Simulation (JASSS), 20(3), 15.

    Google Scholar 

  • Hempel, C. G., & Oppenheim, P. (1948). Studies in the logic of explanation. Philosophy of Science, 15, 135–175.

    Article  Google Scholar 

  • Hogg, M. A., Turner, J. C., & Davidson, B. (1990). Polarized norms and social frames of reference: A test of the self-categorization theory of group polarization. Basic and Applied Social Psychology, 11(1), 77–100.

    Article  Google Scholar 

  • Huckfeldt, R., Johnson, P. E., & Sprague, J. (2004). Political disagreement. The survival of diverse opinions within communication networks. Cambridge University Press.

    Google Scholar 

  • Iyengar, S., & Hahn, K. S. (2009). Red media, blue media: Evidence of ideological selectivity in media use. Journal of Communication, 59(1), 19–39.

    Article  Google Scholar 

  • Keijzer, M. A., Mäs, M., & Flache, A. (2018). Online social networks foster cultural isolation. Groningen.

    Google Scholar 

  • Lau, D. C., & Keith Murnighan, J. (2005). Interactions within groups and subgroups: The effects of demographic faultlines. Academy of Management Journal, 48(4), 645–659.

    Google Scholar 

  • Lazarsfeld, P. F., & Merton, R. K. (1954). Friendship and social process: A substantive and methodological analysis. In M. Berger, T. Abel, & C. H. Page (Eds.), Freedom and Control in Modern Society (pp. 18–66). New York, Toronto, London: Van Nostrand.

    Google Scholar 

  • Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabasi, A. L., Brewer, D., et al. (2009). Computational social science. Science, 323(5915), 721–723.

    Article  Google Scholar 

  • Lewis, K., Gonzalez, M., & Kaufman, J. (2012). Social selection and peer influence in an online social network. Proceedings of the National Academy of Sciences of the United States of America, 109(1), 68–72.

    Article  Google Scholar 

  • Liu, C. C., & Srivastava, S. B. (2015). Pulling closer and moving apart: Interaction, identity, and influence in the U.S. Senate, 1973 to 2009. American Sociological Review, 80(1), 192–217.

    Article  Google Scholar 

  • Loehlin, J. C., & Alexander Beaujean, A. (2016). Latent variable models: An introduction to factor, path, and structural equation analysis (5th ed.). New York and London: Taylor & Francis.

    Google Scholar 

  • Lorenz, J. (2005). A stabilization theorem for dynamics of continuous opinions. Physica a-Statistical Mechanics and Its Applications, 355(1), 217–223.

    Article  MathSciNet  Google Scholar 

  • Lorenz, J. (2007). Continuous opinion dynamics under bounded confidence: A survey. International Journal of Modern Physics C, 18(12), 1819–1838.

    Article  MATH  Google Scholar 

  • Lorenz, J. (2017). Modeling the evolution of ideological landscapes through opinion dynamics. In W. Jager, R. Verbrugge, A. Flache, G. de Roo, L. Hoogduin, & C. Hemelrijk (Eds.), Advances in social simulation 2015 (pp. 255–266). Springer International Publishing.

    Google Scholar 

  • Macy, M. W., Kitts, J., Flache, A., & Benard, S. (2003). Polarization and dynamic networks. A hopfield model of emergent structure. In R. Breiger, K. Carley, & P. Pattison (Eds.), Dynamic Social Network Modeling and Analysis: Workshop Summary and Papers (pp. 162–173). Washington, DC: The National Academies Press.

    Google Scholar 

  • Mark, N. P. (2003). Culture and competition: Homophily and distancing explanations for cultural niches. American Sociological Review, 68(3), 319–345.

    Article  Google Scholar 

  • Marsden, P. V., & Friedkin, N. E. (1993). Network studies of social-influence. Sociological Methods & Research, 22(1), 127–151.

    Article  Google Scholar 

  • Mäs, M., & Flache, A. (2013). Differentiation without distancing. Explaining opinion bi-polarization without assuming negative influence. Plos One 8(11).

    Google Scholar 

  • Mäs, M., & Bischofberger, L. (2015). Will the personalization of online social networks foster opinion polarization. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2553436.

  • Mäs, M., Flache, A., & Helbing, D. (2010). Individualization as driving force of clustering phenomena in humans. PLoS Computational Biology, 6(10), e1000959.

    Article  Google Scholar 

  • Mäs, M., Flache, A., Takács, K., & Jehn, K. A. (2013). In the short term we divide, in the long term we unite: Demographic crisscrossing and the effects of faultlines on subgroup polarization. Organization Science, 24(3), 716–736.

    Google Scholar 

  • Mäs, M., Flache, A., & Kitts, J. A. (2014). cultural integration and differentiation in groups and organizations. In V. Dignum & F. Dignum (Eds.), Perspectives on culture and agent-based simulations. Cham: Springer.

    Google Scholar 

  • Mason, W. A., Conrey, F. R., & Smith, E. R. (2007). Situating social influence processes: Dynamic, multidirectional flows of influence within social networks. Personality and Social Psychology Review, 11(3), 279–300.

    Article  Google Scholar 

  • McPherson, M., Smith-Lovin, L., & Cook, J. M. (2001). Birds of a feather: Homophily in social networks. Annual Review of Sociology, 27, 415–444.

    Article  Google Scholar 

  • Merton, R. K. (1957). Social theory and social structure. Ill: Glencoe.

    Google Scholar 

  • Nagel, E. (1979). The structure of science: Problems in the logic of scientific explanation. Indianapolis, Cambridge: Hackett.

    Google Scholar 

  • Opp, K. D. (2009). Theories of political protest and social movements: A multidisciplinary introduction, critique, and synthesis. New York: Routledge.

    Book  Google Scholar 

  • Opp, K. D., & Gern, C. (1993). Dissident groups, personal networks, and spontaneous cooperation-The East-German revolution of 1989. American Sociological Review, 58(5), 659–680.

    Article  Google Scholar 

  • Popper, K. R. (1959). The logic of scientific discovery. London: Hutchinson.

    MATH  Google Scholar 

  • Postmes, T., Spears, R., Sakhel, K., & De Groot, D. (2001). Social influence in computer-mediated communication: The effects of anonymity on group behavior. Personality and Social Psychology Bulletin, 27(10), 1243–1254.

    Article  Google Scholar 

  • Rosenbaum, M. E. (1986). The repulsion hypothesis: On the nondevelopment of relationships. Journal of Personality and Social Psychology, 51(6), 1156–1166.

    Article  Google Scholar 

  • Sakoda, J. M. (1971). The checkerboard model of social interaction. The Journal of Mathematical Sociology, 1(1), 119–132 (Taylor & Francis Group).

    Google Scholar 

  • Salganik, M. J. (2017). Bit by bit: Social research in the digital age.

    Google Scholar 

  • Salzarulo, L. (2006). A continuous opinion dynamics model based on the principle of meta-contrast. Journal of Artificial Societies and Social Simulation, 9(1).

    Google Scholar 

  • Schelling, T. C. (1971). Dynamic models of segregation. Journal of Mathematical Sociology, 1, 143–186.

    Article  MATH  Google Scholar 

  • Slater, M. D. (2007). Reinforcing spirals: The mutual influence of media selectivity and media effects and their impact on individual behavior and social identity. Communication Theory, 17(3), 281–303.

    Article  Google Scholar 

  • Steglich, C., Snijders, T. A. B., & Pearson, M. (2010). Dynamic networks and behavior: Separating selection from influence. Sociological Methodology, 40(1), 329–393.

    Article  Google Scholar 

  • Tajfel, H. (1978). Social categorization, social identity and social comparison. In Differentiation between social groups: Studies in the social psychology of intergroup relations.

    Google Scholar 

  • Tajfel, H., & Turner, J. C. (1986). The social identity theory of intergroup behavior. In S. Worchel, & W. G. Austin (Eds.), Psychology of intergroup relations (pp. 7–24). Chicago: Nelson-Hall Publishers.

    Google Scholar 

  • Takács, K., Flache, A., & Mäs, M. (2016). Discrepancy and disliking do not induce negative opinion shifts. PLoS One, 11(6), e0157948.

    Article  Google Scholar 

  • Turner, J. H. (1974). The structure of sociological theory. Chicago, Illinois: The Dorsey Press.

    Google Scholar 

  • Turner, J. H. (1995). Macrodynamics. Toward a theory on the organization of human populations. New Brunswick, N.J: Rutgers University Press.

    Google Scholar 

  • Wimmer, A., & Lewis, K. (2010). Beyond and below racial homophily: ERG models of a friendship network documented on Facebook. American Journal of Sociology, 116(2), 583–642.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Sociology/ICS, University of Groningen, Groningen, The Netherlands

    Michael Mäs

Authors
  1. Michael Mäs
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Mäs .

Editor information

Editors and Affiliations

  1. University of Bern, Bern, Switzerland

    Claus Beisbart

  2. Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

    Nicole J. Saam

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Mäs, M. (2019). Challenges to Simulation Validation in the Social Sciences. A Critical Rationalist Perspective. In: Beisbart, C., Saam, N. (eds) Computer Simulation Validation. Simulation Foundations, Methods and Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-70766-2_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-70766-2_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-70765-5

  • Online ISBN: 978-3-319-70766-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation