Abstract
Experimentation represents today a ‘hot’ topic in computing. If experiments made with the support of computers, such as computer simulations, have received increasing attention from philosophers of science and technology, questions such as “what does it mean to do experiments in computer science and engineering and what are their benefits?” emerged only recently as central in the debate over the disciplinary status of the discipline. In this work we aim at showing, also by means of paradigmatic examples, how the traditional notion of controlled experiment should be revised to take into account a part of the experimental practice in computing along the lines of experimentation as exploration. Taking inspiration from the discussion on exploratory experimentation in the philosophy of science—experimentation that is not theory-driven—we advance the idea of explorative experiments that, although not new, can contribute to enlarge the debate about the nature and role of experimental methods in computing. In order to further refine this concept we recast explorative experiments as socio-technical experiments, that test new technologies in their socio-technical contexts. We suggest that, when experiments are explorative, control should be intended in a posteriori form, in opposition to the a priori form that usually takes place in traditional experimental contexts.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Consider for example autonomous robotics, where it is plausible to think that in the future this social scenario will progressively extend both for the increasing use of autonomous robots in everyday life and for the social implication this use will entail.
References
Amigoni, F., Reggiani, M., & Schiaffonati, V. (2009). An insighful comparison between experiments in mobile robotics and in science. Autonomous Robots, 27(4), 313–325.
Amigoni, F., Schiaffonati, V., & Verdicchio, M. (2014). Good experimental methodologies for autonomous robotics: From theory to practice. In F. Amigoni & V. Schiaffonati (Eds.), Methods and experimental techniques in computer engineering. Springer briefs in applied sciences and technology (pp. 37–53). Berlin: Springer.
Barni, M., Perez-Gonzalez, F., Comesana, P., & Bartoli, G. (2007). Putting reproducible signal processing into practice: A case study in watermarking. In Proceedings of IEEE international conference on acoustics speech and signal processing.
Basili, V. R., & Briand, L. C. (Eds.). (1996). Empirical software engineering: An international journal. Berlin: Springer.
Burian, R. M. (1997). Exploratory experimentation and the role of histochemical techniques in the work of Jean Brachet, 1938–1952. History and Philosophy of the Life Sciences, 19, 27–45.
Burian, R. M. (2007). On microRNA and the need for exploratory experimentation in post-genomic molecular biology. History and Philosophy of Life Sciences, 29(3), 285–311.
Denning, P. J. (1980). What is experimental computer science. Communications of the ACM, 23(10), 543–544.
Denning, P. J. (1981). ACM’s president letter. Performance analysis: Experimental computer science at its best. Communications of the ACM, 24(11), 725–727.
Denning, P. J. (2005). Is computer science science? Communications of the ACM, 48(4), 27–31.
Denning, P. J. (2013). The science in computer science. Communications of the ACM, 56(5), 35–38.
Denning, P. J., & Freeman, P. (2009). Computing’s paradigm. Communications of the ACM, 52(12), 28–30.
Feitelson, D. G. (2006). Experimental computer science: The need for a cultural change. Unpublished manuscript available at http://www.cs.huji.ac.il/~feit/papers/exp05.pdf. Last accessed October 2014.
Feldman, J. A., & Sutherland, W. R. (1979). Rejuvenating experimental computer science. Communications of the ACM, 22(9), 497–502.
Franklin, A. (1986). The neglect of experiment. Cambridge: Cambridge University Press.
Franklin, L. R. (2005). Exploratory experiments. Philosophy of Science, 72, 888–899.
Freeman, P. (2008). Back to experimentation. Communications of the ACM, 51(1), 21–22.
Hacking, I. (1983). Representing and intervening. New York: Cambridge University Press.
Hansson, S. O. (2015). Experiments before science? What science learned from technological experiments. In S. O. Hansson (Ed.), The role of technology in science. Philosophical perspectives. Dordrecht: Springer.
Harrison, W., & Basili, V. R. (1996). Editorial. Empirical Software Engineering, 1(1), 5–10.
Hatleback, E., & Spring, J. (2014). Exploring a mechanistic approach to experimentation in computing. Philosophy and Technology, 27(3), 441–459.
Keller, E. F. (2002). Making sense of life: Explaining biological development with models, metaphors, and machines. Cambridge, MA: Harvard University Press.
Kroes, P. (2015). Experiments on socio-technical systems: The problem of control. Science and Engineering Ethics Special Issue on Experiments, Ethics, and New Technologies. doi:10.1007/s11948-015-9634-4.
Langley, P. (1988). Machine learning as an experimental science. Machine Learning, 3, 5–8.
Mayer, B., & Nordio, M. (Eds.). (2010). Empirical software engineering and verification. Lecture notes in computer science. Berlin: Springer.
McCracken, D. D., Denning, P. J., & Brandin, D. H. (1979). An ACM executive committee position on the crisis in experimental computer science. Communications of the ACM, 22(9), 503–504.
McLeod, M., & Nersessian, N. J. (2013). Building simulations from the ground up: Modeling and theory in systems biology. Philosophy of Science, 80(4), 533–556.
Morrison, C., & Snodgrass, R. (2011). Computer science can use more science. Communications of the ACM, 54(6), 38–43.
Newell, A., & Simon, H. (1976). Computer science as empirical inquiry: Symbols and search. Communications of the ACM, 19(3), 113–126.
Radder, H. (Ed.). (2003). The philosophy of scientific experimentation. Pittsburgh: The University of Pittsburgh Press.
Radder, H. (2009). The philosophy of scientific experimentation: A review. Automated Experimentation, 1, 2. doi:10.1186/1759-4499-1-2.
Schiaffonati, V., & Verdicchio, M. (2014). Computing and experiments. Philosophy and Technology, 27(3), 359–376.
Snir, M. (2011). Computer and information science and engineering: One discipline, many specialties. Communications of the ACM, 54(3), 38–43.
Steinle, F. (1997). Entering new fields: Exploratory uses of experimentation. Philosophy of Science, 64, S65–S67.
Tedre, M. (2011). Computing as a science: A survey of computing viewpoints. Minds and Machines, 21, 361–387.
Tedre, M. (2015). The science of computing. Boca Raton: CRC Press, Taylor and Francis Group.
Tichy, W. (1998). Should computer scientists experiment more? IEEE Computer, 31(5), 32–40.
Van de Poel, I. (forthcoming). Society as a laboratory to experiment with new technologies. In E. Stokes, D. Bowman, & A. Rip (Eds.), Embedding and governing new technologies. Singapore: Pan Stanford Publishing.
Vandewalle, P., Kovacevic, J., & Vetterli, M. (2009). Reproducible research in signal processing. IEEE Signal Processing Magazine, 37, 37–47.
Vermaas, P., Kroes, P., van de Poel, I., Franssen, M., & Houkes, W. (2011). A philosophy of technology. From technical artefacts to sociotechnical systems. USA: Morgan and Claypool.
Vincenti, W. (1990). What engineers know and how they know it. Baltimore: The Johns Hopkins University Press.
Vokac, M., Tichy, W., Sjøberg, D. I., Arisholm, E., & Aldrin, M. (2014). A controlled experiment comparing the maintainability of programs designed with and without design patterns—A replication in a real programming environment. Empirical Software Engineering, 9, 149–195.
Waters, C. K. (2007). The nature and context of exploratory experimentation. History and Philosophy of the Life Sciences, 19, 275–284.
Winsberg, E. (2010). Science in the age of computer simulations. Chicago and London: The University of Chicago Press.
Winsberg, E. (2013). Computer simulations in science. In E. N. Zalta (Ed.), The Stanford encyclopedia of philosophy. http://plato.stanford.edu/archives/sum2013/entries/simulations-science/.
Acknowledgments
I am grateful to all the participants and organizers of the workshop “New Technologies as Social Experiments”, held at TU Delft in January 2014, and two anonymous reviewers.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Schiaffonati, V. Stretching the Traditional Notion of Experiment in Computing: Explorative Experiments. Sci Eng Ethics 22, 647–665 (2016). https://doi.org/10.1007/s11948-015-9655-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11948-015-9655-z