Cognitive Metascience: A New Approach to the Study of Theories

Marcin Miłkowski

Polish Academy of Sciences, Institute of Philosophy and Sociology
https://orcid.org/0000-0001-7646-5742


Abstrakt

In light of the recent credibility crisis in psychology, this paper argues for a greater emphasis on theorizing in scientific research. Although reliable experimental evidence, preregistration, methodological rigor, and new computational frameworks for modeling are important, scientific progress also relies on properly functioning theories. However, the current understanding of the role of theorizing in psychology is lacking, which may lead to future crises. Theories should not be viewed as mere speculations or simple inductive generalizations.

To address this issue, the author introduces a framework called “cognitive metascience,” which studies the processes and results of evaluating scientific practice. This study should proceed both qualitatively, as in traditional science and technology studies and cognitive science, and quantitatively, by analyzing scientific discourse using language technology.

By analyzing theories as cognitive artifacts that support cognitive tasks, this paper aims to shed more light on their nature. This perspective reveals that multiple distinct theories serve entirely different roles, and studying these roles, along with their epistemic vices and virtues, can provide insight into how theorizing should proceed. The author urges a change in research culture to appreciate the variety of distinct theories and to systematically advance scientific progress. 


Słowa kluczowe:

theory crisis, cognitive metascience, cognitive artifact, theoretical virtue, epistemic criteria


Afeltowicz, Ł., & Wachowski, W. (2015). How Far we Can Go Without Looking Under the Skin: The Bounds of Cognitive Science. Studies in Logic, Grammar and Rhetoric, 40(1), 91–109. https://doi.org/ 10.1515/slgr-2015-0005   Google Scholar

Almaatouq, A., Griffiths, T. L., Suchow, J. W., Whiting, M. E., Evans, J., & Watts, D. J. (2022). Beyond Playing 20 Questions with Nature: Integrative Experiment Design in the Social and Behavioral Sciences. Behavioral and Brain Sciences, 1–55. https://doi.org/ 10.1017/S0140525X22002874   Google Scholar

American Psychiatric Association. (1980). Diagnostic and statistical manual of mental disorders (3rd ed.). American Psychiatric Association.   Google Scholar

Anderson, J. R. (2007). How Can the Mind Occur in the Physical Universe? Oxford University Press.   Google Scholar

Aronova, E., Oertzen, C. von, & Sepkoski, D. (Eds.). (2017). Data histories. University of Chicago Press.   Google Scholar

Autzen, B. (2021). Is the replication crisis a base-rate fallacy? Theoretical Medicine and Bioethics, 42(5), 233–243. https://doi.org/ 10.1007/s11017-022-09561-8   Google Scholar

Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature News, 533(7604), 452. https://doi.org/ 10.1038/533452a   Google Scholar

Baumeister, R. F., & Tice, D. M. (2022). Ego Depletion is the Best Replicated Finding in All of Social Psychology. Scholarly Journal of Psychology and Behavioral Sciences, 6(2), 686–688. https://doi.org/ 10.32474/SJPBS.2021.06.000234   Google Scholar

Bird, A. (2021). Understanding the Replication Crisis as a Base Rate Fallacy. The British Journal for the Philosophy of Science, 72(4), 965–993. https://doi.org/ 10.1093/bjps/axy051   Google Scholar

Boekel, W., Wagenmakers, E.-J., Belay, L., Verhagen, J., Brown, S., & Forstmann, B. U. (2015). A purely confirmatory replication study of structural brain-behavior correlations. Cortex, 66, 115–133. https://doi.org/ 10.1016/j.cortex.2014.11.019   Google Scholar

Bogen, J., & Woodward, J. (1988). Saving the Phenomena. The Philosophical Review, 97(3), 303. https://doi.org/ 10.2307/2185445   Google Scholar

Borsboom, D. (2008). Psychometric perspectives on diagnostic systems. Journal of Clinical Psychology, 64(9), 1089–1108. https://doi.org/ 10.1002/jclp.20503   Google Scholar

Borsboom, D., Cramer, A., & Kalis, A. (2018). Brain disorders? Not really… Why network structures block reductionism in psychopathology research. Behavioral and Brain Sciences, 1–54. https://doi.org/ 10.1017/S0140525X17002266   Google Scholar

Borsboom, D., & Cramer, A. O. J. (2013). Network Analysis: An Integrative Approach to the Structure of Psychopathology. Annual Review of Clinical Psychology, 9(1), 91–121. https://doi.org/ 10.1146/annurev-clinpsy-050212-185608   Google Scholar

Botvinik-Nezer, R., Holzmeister, F., Camerer, C. F., Dreber, A., Huber, J., Johannesson, M., … Schonberg, T. (2020). Variability in the analysis of a single neuroimaging dataset by many teams. Nature, 582(7810), 84–88. https://doi.org/ 10.1038/s41586-020-2314-9   Google Scholar

Bower, G. H. (1993). The fragmentation of psychology? American Psychologist, 48(8), 905–907. (1994-00003-001). https://doi.org/ 10.1037/0003-066X.48.8.905   Google Scholar

Bringmann, L. F., & Eronen, M. Ilkka. (2016). Heating up the measurement debate: What psychologists can learn from the history of physics. Theory & Psychology, 26(1), 27–43. https://doi.org/ 10.1177/0959354315617253   Google Scholar

Broadbent, A. (2018). Prediction, Understanding, and Medicine. The Journal of Medicine and Philosophy: A Forum for Bioethics and Philosophy of Medicine, 43(3), 289–305. https://doi.org/ 10.1093/jmp/jhy003   Google Scholar

Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365. https://doi.org/ 10.1038/nrn3475   Google Scholar

Cahalan, S. (2019). The Great Pretender. Grand Central Publishers.   Google Scholar

Callebaut, W. (1993). Taking the naturalistic turn or how real philosophy of science is done. University of Chicago Press.   Google Scholar

Callebaut, W. (2013). Naturalizing Theorizing: Beyond a Theory of Biological Theories. Biological Theory, 7(4), 413–429. https://doi.org/ 10.1007/s13752-013-0122-2   Google Scholar

Carsel, T., Demos, A. P., & Motyl, M. (2018). Strong scientific theorizing is needed to improve replicability in psychological science. Behavioral and Brain Sciences, 41, e123. https://doi.org/ 10.1017/S0140525X1800078X   Google Scholar

Chang, H. (2017). VI—Operational Coherence as the Source of Truth. Proceedings of the Aristotelian Society, 117(2), 103–122. https://doi.org/ 10.1093/arisoc/aox004   Google Scholar

Chomsky, N. (1959). Review of Verbal Behavior by B. F. Skinner. Language, 35(1), 26–58.   Google Scholar

Cooper, R. P., & Shallice, T. (1995). Soar and the case for unified theories of cognition. Cognition, 55(2), 115–149. https://doi.org/ 10.1016/0010-0277(94)00644-Z   Google Scholar

Cowan, N., Belletier, C., Doherty, J. M., Jaroslawska, A. J., Rhodes, S., Forsberg, A., … Logie, R. H. (2020). How Do Scientific Views Change? Notes From an Extended Adversarial Collaboration. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 15(4), 1011–1025. https://doi.org/ 10.1177/1745691620906415   Google Scholar

Craver, C. F. (2007). Explaining the Brain: Mechanisms and the Mosaic Unity of Neuroscience. Oxford University Press.   Google Scholar

Craver, C. F. (2009). Mechanisms and natural kinds. Philosophical Psychology, 22(5), 575–594. https://doi.org/ 10.1080/09515080903238930   Google Scholar

Cummins, R. (2000). “How does it work” versus “what are the laws?”: Two conceptions of psychological explanation. In F. Keil & R. A. Wilson (Eds.), Explanation and Cognition (pp. 117–145). MIT Press.   Google Scholar

Dale, R., Dietrich, E., & Chemero, A. (2009). Explanatory Pluralism in Cognitive Science. Cognitive Science, 33(5), 739–742. https://doi.org/ 10.1111/j.1551-6709.2009.01042.x   Google Scholar

Dang, J. (2016). Commentary: A Multilab Preregistered Replication of the Ego-Depletion Effect. Frontiers in Psychology, 7, 1155. https://doi.org/ 10.3389/fpsyg.2016.01155   Google Scholar

Del Pin, S. H., Skóra, Z., Sandberg, K., Overgaard, M., & Wierzchoń, M. (2021). Comparing theories of consciousness: Why it matters and how to do it. Neuroscience of Consciousness, 2021(2), niab019. https://doi.org/ 10.1093/nc/niab019   Google Scholar

Dellsén, F. (2020). The epistemic impact of theorizing: Generation bias implies evaluation bias. Philosophical Studies, 177(12), 3661–3678. https://doi.org/ 10.1007/s11098-019-01387-w   Google Scholar

Di Nardo, P. A., O’Brien, G. T., Barlow, D. H., Waddell, M. T., & Blanchard, E. B. (1983). Reliability of DSM-III Anxiety Disorder Categories Using a New Structured Interview. Archives of General Psychiatry, 40(10), 1070–1074. https://doi.org/ 10.1001/archpsyc.1983.01790090032005   Google Scholar

Dror, I. E., & Gallogly, D. P. (1999). Computational analyses in cognitive neuroscience: In defense of biological implausibility. Psychonomic Bulletin & Review, 6(2), 173–182. https://doi.org/ 10.3758/BF03212325   Google Scholar

Erdin, H. O. (2021). Appraisal of certain methodologies in cognitive science based on Lakatos’s methodology of scientific research programmes. Synthese, 199, 89–112. https://doi.org/ 10.1007/s11229-020-02612-4   Google Scholar

Eronen, M. I., & Bringmann, L. F. (2021). The Theory Crisis in Psychology: How to Move Forward. Perspectives on Psychological Science, 16(4), 779–788. https://doi.org/ 10.1177/1745691620970586   Google Scholar

Ferguson, C. J., & Heene, M. (2012). A Vast Graveyard of Undead Theories: Publication Bias and Psychological Science’s Aversion to the Null. Perspectives on Psychological Science, 7(6), 555–561. https://doi.org/ 10.1177/1745691612459059   Google Scholar

Fiedler, K. (1991). Heuristics and Biases in Theory Formation: On the Cognitive Processes of those Concerned with Cognitive Processes. Theory & Psychology, 1(4), 407–430. https://doi.org/ 10.1177/0959354391014002   Google Scholar

Fiedler, K. (2017). What Constitutes Strong Psychological Science? The (Neglected) Role of Diagnosticity and A Priori Theorizing: Perspectives on Psychological Science, 12(1), 46–61. https://doi.org/ 10.1177/1745691616654458   Google Scholar

Flis, I. (2019). Psychologists psychologizing scientific psychology: An epistemological reading of the replication crisis. Theory & Psychology, 29(2), 158–181. https://doi.org/ 10.1177/0959354319835322   Google Scholar

Frankenhuis, W. E., Panchanathan, K., & Smaldino, P. E. (2022). Strategic ambiguity in the social sciences. Social Psychological Bulletin. https://www.psycharchives.org/en/item/e5bb9192-80a4-4ae4-9cda-5d144008196e   Google Scholar

Fried, E. I. (2020). Lack of Theory Building and Testing Impedes Progress in The Factor and Network Literature. Psychological Inquiry, 31(4), 271–288. https://doi.org/ 10.1080/1047840X.2020.1853461   Google Scholar

Frixione, M. (2001). Tractable competence. Minds and Machines, 11, 379–397.   Google Scholar

Gaj, N. (2016). Unity and Fragmentation in Psychology: The Philosophical and Methodological Roots of the Discipline. Taylor & Francis Ltd.   Google Scholar

Giere, R. N., & Moffatt, B. (2003). Distributed Cognition: Where the Cognitive and the Social Merge. Social Studies of Science, 33(2), 301–310. https://doi.org/ 10.1177/03063127030332017   Google Scholar

Gigerenzer, G. (1991). From tools to theories: A heuristic of discovery in cognitive psychology. Psychological Review, 98(2), 254–267. https://doi.org/ 10.1037/0033-295X.98.2.254   Google Scholar

Gigerenzer, G. (1992). Discovery in Cognitive Psychology: New Tools Inspire New Theories. Science in Context, 5(2), 329–350. https://doi.org/ 10.1017/S0269889700001216   Google Scholar

Gigerenzer, G. (1998). Surrogates for Theories. Theory & Psychology, 8(2), 195–204. https://doi.org/ 10.1177/0959354398082006   Google Scholar

Gitelman, L. (Ed.). (2013). “Raw data” is an oxymoron. The MIT Press.   Google Scholar

Goertzen, J. R. (2008). On the Possibility of Unification: The Reality and Nature of the Crisis in Psychology. Theory & Psychology, 18(6), 829–852. https://doi.org/ 10.1177/0959354308097260   Google Scholar

Gorelick, R. (2011). What is theory? Ideas in Ecology and Evolution, 4, 1–10. https://doi.org/ 10.4033/iee.2011.4.1.c   Google Scholar

Greenwald, A. G. (2012). There Is Nothing So Theoretical as a Good Method: Perspectives on Psychological Science, 7(2), 99–108. https://doi.org/ 10.1177/1745691611434210   Google Scholar

Greenwald, A. G., Pratkanis, A. R., Leippe, M. R., & Baumgardner, M. H. (1986). Under what conditions does theory obstruct research progress? Psychological Review, 93(2), 216–229. https://doi.org/ 10.1037/0033-295X.93.2.216   Google Scholar

Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H., Anggono, C. O., Batailler, C., Birt, A. R., … Zwienenberg, M. (2016). A Multilab Preregistered Replication of the Ego-Depletion Effect. Perspectives on Psychological Science, 11(4), 546–573. https://doi.org/ 10.1177/1745691616652873   Google Scholar

Hensel, W. M. (2020). Double trouble? The communication dimension of the reproducibility crisis in experimental psychology and neuroscience. European Journal for Philosophy of Science, 10(3), 44. https://doi.org/ 10.1007/s13194-020-00317-6   Google Scholar

Hensel, W. M., Miłkowski, M., & Nowakowski, P. (2022). Without more theory, psychology will be a headless rider. Behavioral and Brain Sciences, 45, e20. https://doi.org/ 10.1017/S0140525X21000212   Google Scholar

Hoyningen-Huene, P. (2013). Systematicity: The Nature of Science. Oxford University Press.   Google Scholar

Hughes, B. M. (2018). Psychology in crisis. Palgrave.   Google Scholar

Hutchins, E. (1995). Cognition in the wild. MIT Press.   Google Scholar

Ioannidis, J. P. A. (2005). Why Most Published Research Findings Are False. PLoS Medicine, 2(8), e124. https://doi.org/ 10.1371/journal.pmed.0020124   Google Scholar

Irvine, E. (2021). The Role of Replication Studies in Theory Building. Perspectives on Psychological Science, 16(4), 844–853. https://doi.org/ 10.1177/1745691620970558   Google Scholar

Isaac, A. M. C. (2019). Epistemic Loops and Measurement Realism. Philosophy of Science, 86(5), 930–941. https://doi.org/ 10.1086/705476   Google Scholar

Ivani, S. (2019). What we (should) talk about when we talk about fruitfulness. European Journal for Philosophy of Science, 9(4), 1–18. https://doi.org/ 10.1007/s13194-018-0231-7   Google Scholar

Kawa, S., & Giordano, J. (2012). A brief historicity of the Diagnostic and Statistical Manual of Mental Disorders: Issues and implications for the future of psychiatric canon and practice. Philosophy, Ethics, and Humanities in Medicine: PEHM, 7, 2. https://doi.org/ 10.1186/1747-5341-7-2   Google Scholar

Keas, M. N. (2018). Systematizing the theoretical virtues. Synthese, 195(6), 2761–2793. https://doi.org/ 10.1007/s11229-017-1355-6   Google Scholar

Kilgarriff, A., Baisa, V., Bušta, J., Jakubíček, M., Kovář, V., Michelfeit, J., … Suchomel, V. (2014). The Sketch Engine: Ten years on. Lexicography, 1, 7–36. https://doi.org/ 10.1007/s40607-014-0009-9   Google Scholar

Klein, S. B. (2014). What can recent replication failures tell us about the theoretical commitments of psychology? Theory & Psychology, 24(3), 326–338. https://doi.org/ 10.1177/0959354314529616   Google Scholar

Koyré, A. (1953). An Experiment in Measurement. Proceedings of the American Philosophical Society, 97(2), 222–237.   Google Scholar

Kuhn, T. S. (1977). The essential tension: Selected studies in scientific tradition and change. The University of Chicago Press.   Google Scholar

Langley, P., Simon, H. A., Bradshaw, G. L., & Żytkow, J. M. (1987). Scientific discovery: Computational explorations of the creative processes. MIT Press.   Google Scholar

Laudan, L. (1984). Science and values: The aims of science and their role in scientific debate. University of California Press.   Google Scholar

Lean, O. M., Rivelli, L., & Pence, C. H. (2021). Digital Literature Analysis for Empirical Philosophy of Science. The British Journal for the Philosophy of Science. https://doi.org/ 10.1086/715049   Google Scholar

Leonelli, S. (2016). Data-centric biology: A philosophical study. The University of Chicago Press.   Google Scholar

Levenstein, D., Alvarez, V. A., Amarasingham, A., Azab, H., Chen, Z. S., Gerkin, R. C., … Redish, A. D. (2023). On the Role of Theory and Modeling in Neuroscience. Journal of Neuroscience, 43(7), 1074–1088. https://doi.org/ 10.1523/JNEUROSCI.1179-22.2022   Google Scholar

Litwin, P., & Miłkowski, M. (2020). Unification by Fiat: Arrested Development of Predictive Processing. Cognitive Science, 44(7), e12867. https://doi.org/ 10.1111/cogs.12867   Google Scholar

Longino, H. E. (1996). Cognitive and Non-Cognitive Values in Science: Rethinking the Dichotomy. In L. H. Nelson & J. Nelson (Eds.), Feminism, Science, and the Philosophy of Science (pp. 39–58). Springer Netherlands. https://doi.org/ 10.1007/978-94-009-1742-2_3   Google Scholar

MacCorquodale, K. (1970). On Chomsky’s review of Skinner’s Verbal Behavior. Journal of the Experimental Analysis of Behavior, 13(1), 83–99. https://doi.org/ 10.1901/jeab.1970.13-83   Google Scholar

Manninen, T., Aćimović, J., Havela, R., Teppola, H., & Linne, M.-L. (2018). Challenges in Reproducibility, Replicability, and Comparability of Computational Models and Tools for Neuronal and Glial Networks, Cells, and Subcellular Structures. Frontiers in Neuroinformatics, 12, 20. https://doi.org/ 10.3389/fninf.2018.00020   Google Scholar

Marr, D. (1982). Vision. W. H. Freeman and Company.   Google Scholar

Matthews, G. (2020). Against consensus: Embracing the disunity of personality theory. Personality and Individual Differences, 152, 109535. https://doi.org/ 10.1016/j.paid.2019.109535   Google Scholar

McMullin, E. (2010). The Virtues of a Good Theory. In The Routledge Companion to Philosophy of Science. Routledge. https://doi.org/ 10.4324/9780203744857.ch53   Google Scholar

Meehl, P. E. (1967). Theory-Testing in Psychology and Physics: A Methodological Paradox. Philosophy of Science, 34(2), 103–115. https://doi.org/ 10.1086/288135   Google Scholar

Miłkowski, M. (2019). Fallible Heuristics and Evaluation of Research Traditions. The Case of Embodied Cognition. Ruch Filozoficzny, 75(2), 223–236. https://doi.org/ 10.12775/RF.2019.031   Google Scholar

Miłkowski, M. (2022). Cognitive Artifacts and Their Virtues in Scientific Practice. Studies in Logic, Grammar and Rhetoric, 67(3), 219–246. https://doi.org/ 10.2478/slgr-2022-0012   Google Scholar

Miłkowski, M., Hensel, W. M., & Hohol, M. (2018). Replicability or reproducibility? On the replication crisis in computational neuroscience and sharing only relevant detail. Journal of Computational Neuroscience, 45(3), 163–172. https://doi.org/ 10.1007/s10827-018-0702-z   Google Scholar

Miłkowski, M., & Litwin, P. (2022). Testable or bust: Theoretical lessons for predictive processing. Synthese, 200(6), 462. https://doi.org/ 10.1007/s11229-022-03891-9   Google Scholar

Mischel, W. (2008). The Toothbrush Problem. APS Observer, 21(11). Retrieved from https://www.psychologicalscience.org/observer/the-toothbrush-problem   Google Scholar

Moretti, F. (2000). Conjectures on World Literature. New Left Review, 1, 54–68.   Google Scholar

Morgan, M. S., & Morrison, M. (1999). Models As Mediators. Cambridge University Press.   Google Scholar

Muthukrishna, M., & Henrich, J. (2019). A problem in theory. Nature Human Behaviour, 3, 221–229. https://doi.org/ 10.1038/s41562-018-0522-1   Google Scholar

Nersessian, N. J. (2008). Creating scientific concepts. MIT Press.   Google Scholar

Newell, A. (1973). You can’t play 20 questions with nature and win: Projective comments on the papers of this symposium. In W. G. Chase (Ed.), Visual information processing (pp. 283–308). Academic Press.   Google Scholar

Newell, A., & Simon, H. A. (1972). Human Problem Solving. Prentice-Hall.   Google Scholar

Nickles, T. (2018). TTT: A Fast Heuristic to New Theories? In D. Danks & E. Ippoliti (Eds.), Building Theories: Heuristics and Hypotheses in Sciences (pp. 169–189). Springer. https://doi.org/ 10.1007/978-3-319-72787-5_9   Google Scholar

Norman, D. A. (1991). Cognitive Artifacts. In J. M. Carroll (Ed.), Designing Interaction: Psychology at the Human-Computer Interface (pp. 17–38). Cambridge University Press.   Google Scholar

Norton, J. D. (2021). The Material Theory of Induction. University of Calgary Press.   Google Scholar

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., … Vazire, S. (2022). Replicability, Robustness, and Reproducibility in Psychological Science. Annual Review of Psychology, 73(1), 719–748. https://doi.org/ 10.1146/annurev-psych-020821-114157   Google Scholar

Oberauer, K., & Lewandowsky, S. (2019). Addressing the theory crisis in psychology. Psychonomic Bulletin & Review, 26(5), 1596–1618. https://doi.org/ 10.3758/s13423-019-01645-2   Google Scholar

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251). https://doi.org/ 10.1126/science.aac4716   Google Scholar

Osbeck, L. M., & Nersessian, N. J. (2014). Situating distributed cognition. Philosophical Psychology, 27(1), 82–97. https://doi.org/ 10.1080/09515089.2013.829384   Google Scholar

Pence, C. H., & Ramsey, G. (2018). How to Do Digital Philosophy of Science. Philosophy of Science, 85(5), 930–941. https://doi.org/ 10.1086/699697   Google Scholar

Piper, A. (2020). Can We Be Wrong? The Problem of Textual Evidence in a Time of Data. Cambridge University Press.   Google Scholar

Poldrack, R. A., Kittur, A., Kalar, D., Miller, E., Seppa, C., Gil, Y., … Bilder, R. M. (2011). The Cognitive Atlas: Toward a Knowledge Foundation for Cognitive Neuroscience. Frontiers in Neuroinformatics, 5. https://doi.org/ 10.3389/fninf.2011.00017   Google Scholar

Rescher, N. (1979). Cognitive systematization: A systems-theoretic approach to a coherentist theory of knowledge. Basil Blackwell.   Google Scholar

Roberts, S., & Pashler, H. (2000). How persuasive is a good fit? A comment on theory testing. Psychological Review, 107(2), 358–358.   Google Scholar

Rosenhan, D. L. (1973). On Being Sane in Insane Places. Science, 179(4070), 250–258. https://doi.org/ 10.1126/science.179.4070.250   Google Scholar

Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/ 10.1037/0033-2909.86.3.638   Google Scholar

Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D. (2020). Why Hypothesis Testers Should Spend Less Time Testing Hypotheses. Perspectives on Psychological Science, 16(4), 744–755. https://doi.org/ 10.1177/1745691620966795   Google Scholar

Schindler, S. (2018). Theoretical virtues in science: Uncovering reality through theory. Cambridge University Press.   Google Scholar

Schooler, J. W. (2014). Metascience could rescue the ‘replication crisis.’ Nature, 515(7525), 9. https://doi.org/ 10.1038/515009a   Google Scholar

Scull, A. (2023). Rosenhan revisited: Successful scientific fraud. History of Psychiatry, 0957154X221150878. https://doi.org/ 10.1177/0957154X221150878   Google Scholar

Shmueli, G. (2010). To Explain or to Predict? Statistical Science, 25(3), 289–310. https://doi.org/ 10.1214/10-STS330   Google Scholar

Shmueli, G., & Koppius, O. R. (2011). Predictive Analytics in Information Systems Research. MIS Quarterly, 35(3), 553. https://doi.org/ 10.2307/23042796   Google Scholar

Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., … Nosek, B. A. (2018). Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science, 1(3), 337–356. https://doi.org/ 10.1177/2515245917747646   Google Scholar

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. https://doi.org/ 10.1177/0956797611417632   Google Scholar

Smaldino, P. E. (2017). Models Are Stupid, and We Need More of Them. In R. R. Vallacher, S. J. Read, & A. Nowak (Eds.), Computational Social Psychology (1st ed., pp. 311–331). Routledge. https://doi.org/ 10.4324/9781315173726-14   Google Scholar

Staats, A. W. (1986). Unified Positivism: A Philosophy for Psychology and the Disunified Sciences. Theoretical & Philosophical Psychology, 6(2), 77–90. https://doi.org/ 10.1037/h0091427   Google Scholar

Sullivan, J. A. (2009). The multiplicity of experimental protocols: A challenge to reductionist and non-reductionist models of the unity of neuroscience. Synthese, 167(3), 511–539. https://doi.org/ 10.1007/s11229-008-9389-4   Google Scholar

Suppes, P. (1962). Models of Data. In E. Nagel, P. Suppes, & A. Tarski (Eds.), Logic, Methodology, and Philosophy of Science: Proceedings of the 1960 International Congress (pp. 252–261). Stanford University Press.   Google Scholar

Szollosi, A., & Donkin, C. (2019). Neglected Sources of Flexibility in Psychological Theories: From Replicability to Good Explanations. Computational Brain & Behavior, 2(3–4), 190–192. https://doi.org/ 10.1007/s42113-019-00045-y   Google Scholar

Thagard, P. (1993). Computational philosophy of science. MIT Press.   Google Scholar

Thagard, P., & Findlay, S. (2012). The cognitive science of science: Explanation, discovery, and conceptual change. MIT Press.   Google Scholar

Trafimow, D., & Earp, B. D. (2016). Badly specified theories are not responsible for the replication crisis in social psychology: Comment on Klein. Theory & Psychology, 26(4), 540–548. https://doi.org/ 10.1177/0959354316637136   Google Scholar

Vadillo, M. A. (2019). Ego depletion may disappear by 2020. Social Psychology, 50, 282–291. https://doi.org/ 10.1027/1864-9335/a000375   Google Scholar

Vadillo, M. A., Gold, N., & Osman, M. (2016). The Bitter Truth About Sugar and Willpower: The Limited Evidential Value of the Glucose Model of Ego Depletion. Psychological Science, 27(9), 1207–1214. https://doi.org/ 10.1177/0956797616654911   Google Scholar

Van Rooij, I. (2008). The Tractable Cognition Thesis. Cognitive Science, 32(6), 939–984. https://doi.org/ 10.1080/03640210801897856   Google Scholar

Van Rooij, I., & Baggio, G. (2021). Theory Before the Test: How to Build High-Verisimilitude Explanatory Theories in Psychological Science. Perspectives on Psychological Science, 16(4), 682–697. https://doi.org/ 10.1177/1745691620970604   Google Scholar

Vohs, K. D., Schmeichel, B. J., Lohmann, S., Gronau, Q. F., Finley, A. J., Ainsworth, S. E., … Albarracín, D. (2021). A Multisite Preregistered Paradigmatic Test of the Ego-Depletion Effect. Psychological Science. (Sage CA: Los Angeles, CA). https://doi.org/ 10.1177/0956797621989733   Google Scholar

Wilson, M. (1993). DSM-III and the transformation of American psychiatry: A history. The American Journal of Psychiatry, 150(3), 399–410. https://doi.org/ 10.1176/ajp.150.3.399   Google Scholar

Yarkoni, T. (2022). The generalizability crisis. Behavioral and Brain Sciences, 45, e1. https://doi.org/ 10.1017/S0140525X20001685   Google Scholar

Yarkoni, T., & Westfall, J. (2017). Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning. Perspectives on Psychological Science, 12(6), 1100–1122. https://doi.org/ 10.1177/1745691617693393   Google Scholar

Young, G. (2016). Unifying Causality and Psychology. Springer International Publishing. https://doi.org/ 10.1007/978-3-319-24094-7   Google Scholar

Zhang, J., & Norman, D. A. (1994). Representations in Distributed Cognitive Tasks. Cognitive Science, 18(1), 87–122. https://doi.org/ 10.1207/s15516709cog1801_3   Google Scholar

Zittoun, T., Gillespie, A., & Cornish, F. (2009). Fragmentation or Differentiation: Questioning the Crisis in Psychology. Integrative Psychological and Behavioral Science, 43(2), 104–115. https://doi.org/ 10.1007/s12124-008-9083-6   Google Scholar


Opublikowane
2023-10-26

Cited By /
Share

Miłkowski, M. (2023). Cognitive Metascience: A New Approach to the Study of Theories. Przegląd Psychologiczny, 66(1), 185–207. https://doi.org/10.31648/przegldpsychologiczny.9682

Marcin Miłkowski 
Polish Academy of Sciences, Institute of Philosophy and Sociology
https://orcid.org/0000-0001-7646-5742