Replicability or reproducibility? On the replication crisis in computational neuroscience and sharing only relevant detail
Journal of Computational Neuroscience 3 (45):163-172 (2018)
AbstractReplicability and reproducibility of computational models has been somewhat understudied by “the replication movement.” In this paper, we draw on methodological studies into the replicability of psychological experiments and on the mechanistic account of explanation to analyze the functions of model replications and model reproductions in computational neuroscience. We contend that model replicability, or independent researchers' ability to obtain the same output using original code and data, and model reproducibility, or independent researchers' ability to recreate a model without original code, serve different functions and fail for different reasons. This means that measures designed to improve model replicability may not enhance (and, in some cases, may actually damage) model reproducibility. We claim that although both are undesirable, low model reproducibility poses more of a threat to long-term scientific progress than low model replicability. In our opinion, low model reproducibility stems mostly from authors' omitting to provide crucial information in scientific papers and we stress that sharing all computer code and data is not a solution. Reports of computational studies should remain selective and include all and only relevant bits of code.
Similar books and articles
A Mechanistic Account of Computational Explanation in Cognitive Science and Computational Neuroscience.Marcin Miłkowski - 2016 - In Vincent C. Müller (ed.), Computing and Philosophy. Springer. pp. 191-205.
Explanation and description in computational neuroscience.David Michael Kaplan - 2011 - Synthese 183 (3):339-373.
Replication, replication and replication: Some hard lessons from model alignment.Bruce Edmonds - unknown
Replicability of Experiment.John D. Norton - 2015 - Theoria: Revista de Teoría, Historia y Fundamentos de la Ciencia 30 (2):229.
A New Account of Replication in the Experimental Life Sciences.Stephan Guttinger - 2019 - Philosophy of Science 86 (3):453-471.
From Computer Metaphor to Computational Modeling: The Evolution of Computationalism.Marcin Miłkowski - 2018 - Minds and Machines 28 (3):515-541.
An a priori solution to the replication crisis.David Trafimow - 2018 - Philosophical Psychology 31 (8):1188-1214.
The semantic challenge to computational neuroscience.Rick Grush - 2001 - In Peter K. Machamer, Peter McLaughlin & Rick Grush (eds.), Theory and Method in the Neurosciences. University of Pittsburgh Press. pp. 155--172.
Experimental Reproducibility and the Experimenters' Regress.Hans Radder - 1992 - PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association 1992:63 - 73.
Modeling a theory without a model theory, or, computational modeling “after feyerabend”.Arthur M. Jacobs & Jonathan Grainger - 1999 - Behavioral and Brain Sciences 22 (1):46-47.
Minimal models and canonical neural computations: the distinctness of computational explanation in neuroscience.M. Chirimuuta - 2014 - Synthese 191 (2):127-153.
Explanation in Computational Neuroscience: Causal and Non-causal.M. Chirimuuta - 2018 - British Journal for the Philosophy of Science 69 (3):849-880.
Added to PP
Historical graph of downloads
Citations of this work
Western Skeptic vs Indian Realist. Cross-Cultural Differences in Zebra Case Intuitions.Krzysztof Sękowski, Adrian Ziółkowski & Maciej Tarnowski - forthcoming - Review of Philosophy and Psychology:1-23.
Double trouble? The communication dimension of the reproducibility crisis in experimental psychology and neuroscience.Witold M. Hensel - 2020 - European Journal for Philosophy of Science 10 (3):1-22.
Kinds of Replicability: Different Terms and Different Functions.Vera Matarese - 2022 - Axiomathes 32 (2):647-670.
Cognitive Artifacts and Their Virtues in Scientific Practice.Marcin Miłkowski - 2022 - Studies in Logic, Grammar and Rhetoric 67 (3):219-246.
References found in this work
The Structure of Scientific Revolutions.Thomas Samuel Kuhn - 1962 - Chicago: University of Chicago Press.
Explaining the Brain: Mechanisms and the Mosaic Unity of Neuroscience.Carl F. Craver - 2007 - Oxford University Press, Clarendon Press.
Progress and its Problems: Toward a Theory of Scientific Growth.Larry Laudan - 1977 - University of California Press.
Thinking about mechanisms.Peter Machamer, Lindley Darden & Carl F. Craver - 2000 - Philosophy of Science 67 (1):1-25.