Abstract
Papers in experimental philosophy rarely offer an account of what it would take to reveal a philosophically significant effect. In part, this is because experimental philosophers tend to pay insufficient attention to the hierarchy of models that would be required to justify interpretations of their data; as a result, some of their most exciting claims fail as explanations. But this does not impugn experimental philosophy. My aim is to show that experimental philosophy could be made more successful by developing, articulating, and advancing plausible models of the data that are collected and the analyses that are employed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Some studies examine reaction times in categorization tasks, acceptability judgments, or use paraphrase tasks to uncover tacit or implicit knowledge. Unique problems arise in evaluating each type of data, and the nature of those problems is unique to each methodology. I hope my worries are clear enough that they can be generalized to these other cases where doing so is appropriate.
Thanks are due to an anonymous referee at Philosophical Studies, who asked me to clarify the arguments in the following two paragraphs.
Thanks are due to J. Brendan Richie for help with the framing of these issues.
Function, Object: F(2, 241) = 5.02, p < .01; there was no significant correlation between affective valence and the ability to smell an object, r(251) = .066, p = .30. Biowaste versus Smoothie, U(40) = 94.0, Z = 3.342, p < .001. They also found a significant interaction between function, complexity, and object, which they take to be irrelevant to the hypotheses under consideration: Function, Complexity, Object, F(2, 241) = 4.67, p < .01. Buckwalter and Phelan do not explain why they ignore this interaction, but it is worth noting that ANOVAs can reveal significant main-effects that are meaningful as higher-level expressions of the interaction between multiple variables; where this happens, interpretations based solely on the main-effect are likely to be false or misleading. Buckwalter and Phelan should have investigated the structure of thee three-way interaction (using post hoc tests and adjusting their significance-level to avoid false-positives).
Function, complexity, object, F(2, 241) = 4.67, p < .01.
A linear regression provides a conditional probability distribution for values of a dependent variable, given the relevant independent variables; a regression line expresses the predicted values of a dependent variable, given these independent variables. Since the world is a messy place, data are rarely fully predicted by the independent variables. The deviation of a response from the predicted value is called the ‘residual’, and R2 is calculated by subtracting the residual variability of a model from 1 to yield a measure of the variance that is explained by each independent variable. Ideally, R2 measures the correlation between the value of an independent variable and the value of a dependent variable. Where they are perfectly correlated, R2 = 1; where there is no relationship between them, R2 = 0.
Warmth (R2 = .05, F = 6.6, p = .011); philosophical expertise (R2 = .14, F = 12.4, p = .001).
They found a moderate negative relationship between visual cognitive style and utilitarian judgment, r(49) = −.37, p = .007. This effect was stable across plausible types of demographic variation (education; politics; gender; religion).
Thanks to Rik Hine and to an anonymous referee for pushing me to clarify both of these points.
In Experiment 2: F(1, 40) = 7.033, p = .011; F(1, 40) = 6.30, p = .016.
Study 2: t(40) = 2.039, p = .048; t(40) = 2.703, p = .01.
Increasing the number of shocks for a profit: t(40) = 2.195, p = .034; t(40) = 2.027, p = .049. Decreasing shocks at a cost: t(40) = 2.696, p = .01; t(40) = 2.6517, p = .011.
This model correctly predicted 90 % of choices in Experiment 2; Bayesian model comparisons were used to show that this model was favored over a number of alternatives, including more standard models of economic choice behavior.
References
Amit, E., & Greene, J. (2012). You see, the ends don’t justify the means. Psychological Science, 23(8), 861–868.
Bogen, J., & Woodward, J. (1998). Saving the phenomena. The Philosophical Review, 97(3), 303–352.
Buckwalter, W. & M. Phelan (2013). Function and feeling machines. Philosophical Studies, 166(2), 349–361. Online supplementary material http://goo.gl/D27JTg. Accessed 1 March 2015.
Chambers, C. (2012). The dirty dozen: A wish list for psychology and cognitive neuroscience. http://goo.gl/XluVRQ, Accessed 31 January 2014.
Chambers, C. et al (2013). Trust in science would be improved by study pre-registration. The Guardian. http://goo.gl/L1Hzck. Accessed 31 January 2014.
Crockett, M. (2014). Behind the scenes of a ‘shocking’ new study on human altruism. The Guardian. http://gu.com/p/43gpm/stw Accessed 2 December 2014.
Crockett, M., Kurth-Nelsona, Z., Siegela, J., Dayan, P., & Dolan, R. (2014). Harm to others outweighs harm to self in moral decision making. PNAS, 111(48), 17320–17325.
Cummins, R., Roth, M., & Harmon, I. (2014). Why it doesn’t matter to metaphysics what Mary learns. Philosophical Studies, 167(3), 541–555.
Haugeland, J. (1991). Representational genera. In W. Ramsey, S. Stich, & D. Rumelhart (Eds.), Philosophy and connectionist theory (pp. 61–89). Hillsdale: Lawrence Erlbaum Associates.
Nuzzo, R. (2014). Scientific method: Statistical errors. Nature, 506, 150–152.
Rosenthal, D. (2011). Mental quality, valence, and intuition: Comments on Edouard Machery. https://wfs.gc.cuny.edu/DRosenthal/www/DR-MERG.pdf.
Schulz, E., Cokely, E. T., & Feltz, A. (2011). Persistent bias in expert judgments about free will and moral responsibility. Consciousness and Cognition, 20(4), 1722–1731.
Shanahan, K. (2002). A systematic error in mass flow calorimetery demonstrated. Thermochimica Acta, 382(2), 95–100.
Suppes, P. (1962). Models of data. In E. Nagel, P. Suppes, & A. Tarski (Eds.), Logic, methodology and philosophy of science (pp. 252–261). Stanford: Stanford University Press.
Wimsatt, W. (1974). Complexity and organization. In K. Schaffner, & R. Cohen (Eds.), Boston studies in the philosophy of science (Vol. 20, pp. 67–86). Dordrecht: Reidel.
Wimsatt, W. (2007). Re-engineering philosophy for limited beings: Piecewise approximations to reality. Cambridge: Harvard University Press.
Winsberg, E., Huebner, B., & Kukla, R. (in press). Accountability, values, and social modeling in radically collaborative research. Studies in the history and philosophy of science, 46, 16–23.
Acknowledgments
Eric Winsberg and Rebecca Kukla helped me see that the relationship between models of data and scientific explanation was relevant to experimental philosophy. I received helpful feedback on an early version of this paper from Rik Hine and an audience at the Southern Society for Philosophy and Psychology (Austin, 2013). Ruth Kramer, James Mattingly, and J. Brendan Ritchie read drafts of this paper, and offered comments that made the arguments stronger than they otherwise would have been. Finally, I would like to thank all of the anonymous reviewers of this paper; I appreciated the time they took to offer comments, even where I disagreed with them.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Huebner, B. What is a philosophical effect? Models of data in experimental philosophy. Philos Stud 172, 3273–3292 (2015). https://doi.org/10.1007/s11098-015-0469-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11098-015-0469-2