No Cross-Cultural Differences in Gettier Car Case Intuition: A Replication Study of Weinberg et al. 2001 Minsun Kim (Yale University) Yuan Yuan (Yale University) Consider the following case, and the question that follows: Bob has a friend, Jill, who has driven a Buick for many years. Bob therefore thinks that Jill drives an American car. He is not aware, however, that her Buick has recently been stolen, and he is also not aware that Jill has replaced it with a Pontiac, which is a different kind of American car.1 Does Bob really know that Jill drives an American car, or does he only believe it? This case is a variant of classic Gettier cases, in which the subject purportedly has justified true belief but not knowledge. Since their emergence fifty years ago (Gettier 1963), Gettier-style cases have been used by epistemologists to refute the traditional conception of knowledge as justified true belief. As Timothy Williamson notes, "the refutation was accepted almost overnight by the community of analytic epistemologists. [The] thought experiments were found intrinsically compelling." (Williamson 2007, p.180) In "Normativity and Epistemic Intuitions" (NEI), Weinberg, Nichols and Stich argue from empirical data that this denial of knowledge is not universal. The empirical data are drawn from a survey conducted by the authors in which Rutgers University undergraduate students of different ethnic backgrounds were asked to respond to the case above. Whereas the majority of Westerners answered that "Bob only believes," the 1 The case is taken verbatim from Weinberg et al. 2001, p.20. majority of East Asians gave an unorthodox answer: "Bob really knows." We will refer to the discrepancy between the East Asian and Western participants' responses to this case as the Car Case effect. NEI argues that the Car Case effect, along with other similar effects, indicates the existence of systematic cross-cultural difference in epistemic intuitions. NEI then mounts the following attack on the entire practice of using intuitions to guide our epistemological inquiries: since our intuitions about such cases are contingent on arbitrary factors such as ethnicity, intuitions should not be trusted as evidence that bears on the nature of knowledge (Weinberg et al. 2001, p. 14). NEI has generated a firestorm of debate regarding the philosophical implications of the Car Case effect.2 This debate consists of questions which arise once we accept the findings of the NEI study: given that the majority of the East Asian participants do not share the traditional Western intuition, what follows? Some argue that such studies do not reliably elicit robust intuitions, and thus lack philosophical significance (Kauppinen 2007). In a similar vein, some suggest that these results reveal merely verbal divergence rather than substantive disagreement (Sosa forthcoming). A more radical school of thought argues that the existence of cross-cultural differences in intuition would leave the philosophical project untouched, insofar as intuitions contribute nothing to philosophy (Cappelen forthcoming). Our aim in this paper is not to contribute to this philosophical debate over what the implications of the Car Case effect would be if it exists. Our aim is instead to examine whether or not the effect exists at all. If the effect does not exist, the debate over its implications is groundless. Our project fits into a growing concern in the social sciences regarding studies with flashy results failing to replicate (e.g., Doyen et al., 2012). In reading NEI, we noticed 2 Since publication, NEI has been cited 404 times according to Google Scholar. (As of May 1, 2014) that the existence of the Car Case effect was generalized from the responses of 23 East Asian participants. It strikes us as too hasty to draw a conclusion from this sample size. 1. Existing Replication Studies Worries over the small sample size of the NEI study have prompted replication studies by Seyedsayamdost (2014) and Nagel (2013). Neither of them found evidence of the Car Case effect. In each study, the majority of both Westerners and East Asians gave the orthodox answer denying that Bob really knows. Although these studies give us reason to doubt the existence of the Car Case effect, they differ from the NEI study in important ways that detract from the decisiveness of their results. The goal of our study was to stay as close to the original NEI study as possible. Firstly, we used exactly the same case and questions as the NEI Car Case. Seyedsayamdost made changes to the case, 3 while Nagel changed the answer options available to the participants.4 The large number of cases preceding the Car Case in Nagel's study further differentiated it from the original NEI study. The Car Case was the 17th case given to all of Nagel's participants. Secondly, our study used the same data collection methodology as NEI: in-person surveys as opposed to online or laboratory testing, as done by Seyedsayamdost for Data Sets 3 and 4. Thirdly, our participants resembled NEI study in all important aspects, as we surveyed university students in the northeast region of the United States. In comparison, Seyedsayamdost's subjects (including non-University populations) were mainly in Britain, and Nagel conducted her 3 Seyedsayamdost made the following changes to the case in his data set 4: replaced the names of the cars from Buick and Pontiac to Toyota and Honda, respectively and also changed the origin of the cars from 'American' to 'Japanese', accordingly (Seyedsayamdost 2014, p.9). 4 While NEI provided two answer choices, "really knows" and "only believes," Nagel provided three answer choices: "Yes, she knows," or "No, she doesn't know, or "Unclear – not enough information provided in the story." A follow-up question was provided for those who selected the first option (Nagel 2013, p.5). study at a Canadian university. It is plausible that there are some factors distinguishing the East Asian population of the Northeast United States with the East Asian population in Britain or Canada. It is also plausible that such factors influence the way the East Asians of each region respond to the Car Case. In sampling exclusive from the Northeast United States, our study would be immune to such factors were they to exist. If the NEI study picked up on a phenomenon that really exists, ours should detect it as well. Another advantage of our study is our large sample size of East Asians. There were 21 East Asians surveyed in Nagel's study and 42 East Asians surveyed in Seyedsayamdost's Data Set 1, which is the only data set in his study that employed exactly the same case and method as those of NEI. Our sample contains 82 East Asians. An advantage of our larger sample size is that it satisfies the 2.5 sample size rule, according to which replication studies should have sample sizes that are 2.5 times the size of the original study (Simonsohn, 2013).5 2. Methods and Results 2.1. Methods Participants. The participants were 207 Yale University students (48% female). Individuals who completed the survey were offered candy in compensation. Individuals who did not complete the survey were removed, for a total sample size of 202 participants. 40% self-identified as East Asian, and 28% as Caucasian. 70% of participants answered that English was their first language. Materials and Procedure. Our survey consisted of two main components: a knowledge survey and a demographic questionnaire. Participants first completed the knowledge survey, and then completed the demographic questionnaire. The knowledge 5 A sample size 2.5 times the original study's sample is necessary to achieve the typically proposed statistical power for an informative failure to replicate (~80%), if the true effect is 0 (Simonsohn 2013, p. 13). survey was composed of a single vignette, the original Gettier Car Case given above. We used exactly the same wording found in the NEI study. In addition to asking the knowledge question (Does Bob really know that Jill drives an American car, or does he only believe it?), we asked participants how confident they were about their answers, on a scale of 1 to 7. All participants completed a demographic questionnaire that asked them to provide 5 categories of demographic data: (1) Yale college major/intended major, (2) gender, (3) first language, (4) ethnic background, (5) number of philosophy courses taken. Question (4) asked participants to self-identify between six categories: East Asian, Caucasian, South Asian, African American, Latino, and Other. Participants who chose "Other" were asked to write in their own descriptions. 2.2. Results Because we focus on the contrast between East Asian (n=82) and Caucasian (n=58) participants, all analyses are conducted on data from those two groups. The percentage of the participants who answered "only believe" in the two groups was strikingly similar: 85.4% for the East Asian group and 86.2% for the Caucasian group. This difference is not statistically significant, χ2 (1, N=140) = 0.02, p=0.89. We also measured the confidence levels of each participant in his or her answer to the Knowledge question. Following the method first developed by Starmans and Friedman (2012), we define a weighted knowledge ascription as the product of the answer to the dichotomous knowledge question (really knows=1; only believes=-1) and the reported confidence (1 to 7). Thus, scores for this measure fall on a fourteen-point scale, ranging from -7 (fully confident knowledge denial) to 7 (fully confident knowledge ascription). The mean of the weighted knowledge ascription for our East Asian participants is -3.63, while the mean of it for our Caucasian participants is -3.60. Here again, there is no significant difference between the two groups, t (137) = .04, p = .97. If we look at the 95% confidence interval of the weighted knowledge ascription, the entire interval falls within the "only believes" region of the scale, both for Caucasians (4.53 to -2.73) and for East Asians (-4.63 to -2.58).In other words, both groups strongly rejected the notion that Bob really knows. We expect the criticism that the East Asian participants in our study are likely to be second or even third generation immigrants to the U.S., and thus have deeply adapted to the Western culture, so that their answers become homogenous with the Westerners'. Firstly, if this is a legitimate worry, we cannot see how the student population at Rutgers University tested in the NEI study is immune to it. However, our study employs a precautionary step to address this concern. We used participants' first language as a rough indicator of the depth of their Western assimilation. Among our East Asian participants, 87% of those who had English as their first language (n=45) and 84% of those who did not have English as their first language (n=37) answered that "Bob only believes." This difference is not statistically significant χ2 (1, N = 82) = 0.14, p=0.71. 3. Conclusion Our study, conducted using a sample size 2.5 times larger than the NEI study's, resembled the original study in all relevant ways. We found no difference between East Asians and Caucasians' responses to the Car Case. Both overwhelmingly rejected the claim that Bob knows: 85.4% of East Asians and 86.2% of Caucasian group denied knowledge. As the graph below indicates, our results are in good company. The Car Case effect does not exist in any sample other than the NEI study's original sample of 23 East Asians. In all other studies, the overwhelming majority of East Asians denied knowledge. Taken together, the repeated failures of replication give overwhelming evidence that the Car Case effect does not exist. Percentage of participants in NEI and replication studies who chose "Bob really knows" The total body of evidence does not support the existence of an ethnicity effect on knowledge attribution. Indeed, there is cause for doubting the existence of newly discovered demographic effects beyond knowledge attribution. 85.0% of X-Phi studies claiming any kind of demographic effect (non-Gettier-style cases included) fail to replicate, while only 14.3% of X-phi studies not claiming a demographic effect fail to replicate.6 Of course, we cannot rule out the possibility that there is some demographic effect yet to be discovered. If subsequent studies indicate the existence of such a demographic effect, the issue of its philosophical implications warrants serious revisitation. But given the current absence of empirical evidence supporting the existence of such effects, this debate is best left tabled. 6 These figures were calculated based on data presented in Mott 2014. References Adleberg, Toni, Morgan Thompson and Eddy Nahmias forthcoming, "Do Men and Women Have Different Philosophical Intuitions? Further Data."Philosophical Psychology. Buckwalter, Wesley and Stephen Stich 2013, "Gender and Philosophical Intuition." In J. Knob and S. Nichols (eds.), Experimental Philosophy, Vol.2. New York: Oxford University Press. Cappelen, Herman forthcoming, "X-phi without intuitions?" In D. Rowbottom and A. Booth (eds.), Intuitions. New York: Oxford University Press. Doyen, Stéphane, Olivier Klein, Cora-Lise Pichon and Axel Cleeremans 2012,"Behavioral Priming: It's All in the Mind, but Whose Mind?" PLoS ONE, 7(1):e29081. Gettier, Edmund 1963, "Is Justified True Belief Knowledge?" Analysis 23: 121-123. Kauppinen, Antti 2007, "The Rise and Fall of Experimental Philosophy," Philosophical Explorations 10 (2):95 – 118. Mott, Christian 2014, "Replications in Experimental Philosophy," http://pantheon.yale.edu/~jk762/xphipage/Experimental%20PhilosophyReplications.html. Nagel, Jennifer, Valerie San Juan and Raymond A. Mar 2013, "Lay Denial of Knowledge for Justified True Beliefs," Cognition 129: 652-611. Seyedsayamdost, Hamid 2014, On Normativity and Epistemic Intuitions: Failure of Replication. Unpublished draft. Simonsohn, Uri 2013, "Small Telescopes: Detectability and the Evaluation of Replication Results." Available at SSRN: http://dx.doi.org/10.2139/ssrn.2259879 Sosa, Ernest forthcoming, "Experimental Philosophy and Philosophical Intuition." In a special issue of Philosophical Studies. Weinberg, Jonathan, Shaun Nichols and Stephen Stich 2001, "Normativity and Epistemic Intuitions." Philosophical Topics 29 (1&2): 429-460. Williamson, Timothy 2007, The Philosophy of Philosophy. Malden: Blackwell Publishing.