University of Colorado, Boulder CU Scholar Philosophy Graduate 7eses & Dissertations Philosophy Summer 5-29-2014 Intuitive and Reective Responses in Philosophy Nick Byrd University of Colorado Boulder, nick.a.byrd@gmail.com Follow this and additional works at: h9p://scholar.colorado.edu/phil_gradetds Part of the Epistemology Commons 7is 7esis is brought to you for free and open access by Philosophy at CU Scholar. It has been accepted for inclusion in Philosophy Graduate 7eses & Dissertations by an authorized administrator of CU Scholar. For more information, please contact cuscholaradmin@colorado.edu. Recommended Citation Byrd, Nick, "Intuitive and Re8ective Responses in Philosophy" (2014). Philosophy Graduate eses & Dissertations. Paper 6. INTUITIVE AND REFLECTIVE RESPONSES IN PHILOSOPHY by NICK BYRD B.A. Palm Beach Atlantic University, 2009 A thesis submitted to the faculty of the Graduate School of the University of Colorado in partial fulfillment the requirement for the degree of Master of Arts in Philosophy Philosophy Department 2014 This thesis entitled: INTUITIVE AND REFLECTIVE RESPONSES IN PHILOSOPHY written by Nick Byrd has been approved for the Department of Philosophy by Michael Huemer Michael Tooley Robert Rupert The final copy of this thesis has been examined by the signatories, and we find that both the content and the form meet acceptable presentation standards of scholarly work in the above mentioned discipline. IRB protocol #13-0678 Nick Byrd (M.A., Philosophy) INTUITIVE AND REFLECTIVE REASONING IN PHILOSOPHY Committee: Michael Huemer, Robert Rupert, and Michael Tooley Cognitive scientists have revealed systematic errors in human reasoning. There is disagreement about what these errors indicate about human rationality, but one upshot seems clear: human reasoning does not seem to fit traditional views of human rationality. This concern about rationality has made its way through various fields and has recently caught the attention of philosophers. The concern is that if philosophers are prone to systematic errors in reasoning, then the integrity of philosophy would be threatened. In this paper, I present some of the more famous work in cognitive science that has marshaled this concern. Then I present reasons to think that those with training in philosophy will be less prone to certain systematic errors in reasoning. The suggestion is that if philosophers could be shown to be less prone to such errors, then the worries about the integrity of philosophy could be constrained. Then I present evidence that, according to performance on the CRT (Frederick 2005), those who have benefited from training and selection in philosophy are indeed less prone to one kind of systematic error: irrationally arbitrating between intuitive and reflective responses. Nonetheless, philosophers are not entirely immune to this systematic error, and their proclivity for this error is statistically related to their responses to a variety of philosophical questions. So, while the evidence herein puts constraints on the worries about the integrity of philosophy, it by no means eliminates these worries. The conclusion, then, is that the present evidence offers prima facie reasons to ascribe a mitigated privilege to philosophers' ability to rationally arbitrate between intuitive and reflective responses. iii For Hannah Acknowledgements Thanks to Michael Huemer for offering important challenges to the motivations of the project and the arguments throughout. Thanks also to Robert Rupert and Michael Tooley for their helpful comments. Special thanks to Bradley Monton for supporting, encouraging, and providing helpful feedback on my experimental research. I also owe a great deal to various faculty members in cognitive science and psychology. First, thanks to Michael Mozer, Tor Wager, Matt Jones, Rob Rupert, Brian Talbot, and Shaw Ketels for introducing me to some of the literature cited in this thesis and to Chick Judd, Joshua Correll, Jake Westfall, and Katie Wolsiefer for their patient guidance with experimental design and statistical analysis. More thanks are in order to Joe Wilson, Caleb Pickard, Daniel Coren, and Joe Fraley for their help translating some PhilPapers questions (Bourget et al 2009) into intelligible questions for non-philosophers. I also need to thank Brian Leiter for posting a link to a survey on LeiterReports.com, which successfully recruited over 500 participants. Recruiting participants for this research would have been exponentially more challenging without Dr. Leiter's support. Importantly, I owe gratitude to the Institute of Cognitive Science at the University of Colorado at Boulder for providing some of the funding for this project and the Philosophy Department for generously offering the assistantships that contributed to this research. Finally, I would like to thank my wife Hannah for her crucial role in preparing the IRB application, for talking through various parts of the project, and for proofreading and commenting on the manuscript. I also want to express my gratitude more generally: thank you for supporting my academic goals, for kindly humoring my queries, and for accepting my proposal of life-long partnership. v CONTENTS SECTION 1. INTRODUCTION................................................. 1 2. SOME HISTORY.................................................. 3 3. PHILOSOPHICAL REASONING.............................. 7 4. DUAL-PROCESS THEORIES.................................. 11 The Cognitive Reflection Test................................13 Characterizing the Intuitive-Reflective Dichotomy.......15 Summary........................................................19 5. A CORRELATIONAL STUDY..................................20 Introduction.....................................................20 Method.......................................................... 20 Results...........................................................22 Discussion......................................................24 Conclusions.....................................................29 Future Research............................................... 30 6. CONCLUSION.....................................................31 BIBLIOGRAPHY...........................................................33 vi FIGURES Figure 1. Wason Card Selection Task (Wason, 1977).................... 5 2. Example of a CRT question...................................... 13 3. Example of original and adapted PhilPapers question........ 21 4. Comparison of CRT Performance................................25 vii 1. INTRODUCTION Cognitive science in the 20th century did well to, among other things, cast a cloud of doubt over certain ideals about human rationality. Some fields experienced this doubt sooner than others. Perhaps psychologists were the first to seriously countenance their experimental results that highlighted the failures of human reasoning. Later, economists would begin to notice threats to their classical notion of economical rationality- e.g., "economic man" or homo economics (Mill, 1997). In experiment after experiment, the behaviors and judgments of both novices and experts deviated from logical, mathematical, and economical norms. So throughout the 20th century it became increasingly uncontroversial to claim that human reasoning, judgment, learning, and decision-making could no longer be characterized by the classic notions of rationality, which were thoroughly optimistic about humans' ability to transcend the vestigial cognitive habits that threaten to frustrate our most earnest attempts to conform to logical, mathematical, and economical norms. It seems that some philosophers have only recently started taking seriously the various challenges to the idea that philosophers can be ideally analytic, or rational, reasoners. Some philosophers have been so intrigued by these challenges that they have taken to vacating their metaphorical armchairs for the world of experimental psychology (Alfano and Loeb 2014, Knobe et al 2012, Knobe and Nichols 2012, 2013). And as more philosophers have learned about the extensive literature that chal- !1 lenges traditional ideals about rationality-and others have run their own experiments-some concerns about the rationality of philosophical practice have emerged. These concerns rekindled interest in the empirical and conceptual questions surrounding how philosophers reason as well as the normative and metaphilosophical questions about how philosophers should reason. The remainder of this paper is a humble attempt to contribute something to this topic. In Section 2, I will highlight some of the more famous research on the kinds of reasoning that are most akin to philosophical reasoning. Then, in Section 3, I will discuss the nature of philosophical training and selection, show how they have benefitted from philosophical training and selection often outperform others on a variety of reasoning tasks, and characterize a duality in analytic philosophers' reasoning. This will bring us to Section 4 where I will show how this duality in philosophical thinking fits into the larger landscape of dual-process theories in cognitive science. With the help of Shane Frederick's (2005) Cognitive Reflection Test (CRT), I will distill an overarching dichotomy from various dual-process theories that will guide the discussion of reasoning for the remainder of the paper. In Section 5, I will look back to Section 3, where I discuss the nature of training and selection in philosophy, in order to motivate a correlational study and furnish predictions about how philosophers will perform on the CRT, how training and selection in philosophy will affect performance on the CRT, how performance on the CRT will correlate with one's responses to various philosophical questions, and how one's sex will correlate with performance on this test. Then I will discuss the results, the implications of the results, some concerns about the study, and make some suggestions for further research. In Section 6, I will conclude that alleged threats to the integrity of philosophical reasoning can be mitigated, although not entirely eliminated, by the apparently effective training and selection of philosophy departments. This conclusion will by no means resolve the debates about human rationality or the rationality of philosophical practice, but it will do well to offer provisional constraints on the severity of the concern about the rationality of philosophers' appeals to intuition. In the end, concerns about the integrity of philosophy will not be fully distinguished. I will show that while philosophical training and selection might make one significantly less prone to at least one kind of reasoning error, philosophers retain a limited proclivity to error and this proclivity for error is significantly related to some of their philosophical views. !2 This indicates that philosophers' reasoning could still have some kind of debilitating affect on philosophical discourse or progress. 2. SOME HISTORY In order to begin the discussion of the systematic and predictable errors in reasoning, I will introduce the study of statistical reasoning-e.g., probabilistic inference (Bar-Hillel 1973, Chapman and Chapman 1967, 1969, Edwards et al 1968, Epley et al 2004, Epley and Golivoch 2001, 2006, Evans 1977, Galbraith and Underwood 1973, Hogarth 1975, Johnson-Laird et al 1999, Mans 1970, Rips 1975, Savage 1971, Staël von Holstein 1971). Statistical reasoning tasks often required participants to describe something statistically or make an inference under conditions that impose uncertainty. What these tasks reveal is that it is very common for both expert and novice reasoners to make errors in statistical reasoning tasks (for a helpful summary of this kind of research, see Tversky and Kahneman 1974). Interestingly, patterns and similarities manifest in participants' errors, implying that these errors might have been systematic and perhaps symptomatic of a certain cognitive strategy. Tversky and Kahneman (1974) and others have attempted provide an account of these systematic errors by introducing and characterizing these cognitive strategies: "heuristics and biases" (Evans 1984, 1989; Gilovich Griffin and Kahneman 2002, Kahneman et al 2002, Kahneman and Frederick 2005, Levinson 1995, Tversky 1974, Tversky and Kahneman 1973). Heuristics and biases reduce the complexity and higher-level computation required by certain tasks to simpler judgments (Tverksy and Kahneman 1974, 1124). In other words, Kahneman and Tversky claimed that participants recruited heuristics or biases when answering certain statistical questions-as opposed to performing complete calculations-and that the recruitment of these heuristics or biases accounted for the patterns and similarities in participants' errors-e.g., why participants' answers to various statistical questions were apparently insensitive to sample size (ibid., Kahneman and Tversky 1973, Shortliffe and Buchanon 1975). A newcomer to the field of cognitive science might wonder why researchers would waste time speculating about the processes that underly a reasoning task. !3 The newcomer might ask, "Why don't you just ask the reasoner? Wouldn't some reasoners be able to trace the steps of their calculation and figure out where they went wrong?" The short answer is, "No. Not necessarily." The reason is that our conscious access to the cognitive processes that lead to our judgments and behaviors appears to be limited, fallible, or downright non-veridical. In a variety of experiments, most reasoners seemed almost entirely unaware of the cognitive processes that lead to their judgments and behaviors (Goethals and Reckman 1973, Jack and Roepstorff 2002, 2003, Johansson et al 2005, Maier 1931, Nisbett and Bellows 1977, Nisbett and Valins 1972, Nisbett and Wilson 1977a, 1977b, 1978, Rich 1979, Rosenfeld and Baer 1969, Speilberger 1962, Storm and Nisbett 1970, Zell and Krizan 2014; for replies see Forsyth 1980 and Jack 2013; for outright critiques see Lieberman 1979, Petitmengin et al 2013, Smith and Miller 1978, and White 1980). In some experiments, participants would be asked to make a judgment about an object or set of objects. For example, some participants were asked which of four pairs of stockings was the highest in quality (Nisbett and Wilson 1977). Then they were asked to report the process or reasons that lead to their judgment. The vast majority of participants chose the rightmost pair of stockings. Participants provided various reasons for choosing the right-most stocking-e.g., the rightmost pair of stockings seemed more finely knit. In these studies, it can easily be shown that participants' reported reasons were suspect. In the stocking experiment, all four pairs of stockings were identical. It is easy to provide alternative reasons for participants' judgments that do a better job of explaining the experimental outcome than the participants' own reasons. In retail settings, it is apparently common for participants' to associate position with quality, perhaps because differences in price-often perceived as an indicator of quality-are typically arranged in ascending order from left to right. So, in addition to showing a position effect in consumer settings, the stocking experiment helps to demonstrate why there would be widespread doubt among cognitive scientists about introspective access to one's own reasoning processes. Now I can return to statistical reasoning. I have explained why one can doubt that we have access to our own reasoning processes. So when trying to understand why reasoners seem to make systematic errors in statistical reasoning tasks, it simply will not suffice to ask the reasoners !4 where and how they went astray since they will not likely have veridical access to the actual cognitive process involved in the judgment(s). Since the time of these results about statistical reasoning and introspective access to reasoning, other forms of reason have also been studied, and with similar results. In an effort to highlight reasoning that might be isomorphic to philosophical reasoning, I will turn to logical reasoning- e.g., deductive and conditional reasoning-the study of which has revealed further systematic errors in reasoning (Barwise 1993, Bell and Johnson-Laird-1998, Braine 1998, Bonatti 1994, Cummins 1991, Evans 1993, 1998, 2002, Evans Newstead and Byrne 1993, Evans Clibbens and Rood 1996, Evans, Dugan and Revlin 1990, Hardman and Payne 1995, Johnson-Laird 1999, Johnson-Laird and Byrne 2002, Shynkaruk and Thompson 2006, Thompson 2010, Turner and Thompson 2009, Wetherick and Golhooly 1990). A famous example of a logical reasoning task is the Wason selection task (Wason 1977, Evans 1993, Griggs Newstead and Evans 1995, Green Over and Pine 1997). It could be useful to attempt the Wason selection task before continuing. ! In the Wason Selection task, participants are presented with four cards, two of which show numbers and the other two show colors-or letters, depending on the variation. The participants are asked which card(s) they must turn over to confirm or falsify the following claim: if a card shows an even number of one side, then it's opposite face is gray. If one is trained in formal logic, then one might deduce that only cards with both an even number on one face and a non-gray color on the other face are needed to invalidate this rule. Importantly, less than 10% of subjects choose these respective cards (ibid., and "Wason Selection Task"). 3 8 Figure 1. Each card has a number on one side, and a patch of color on the other. Which card(s) must be turned over to test the following claim: if a card shows an even number on one face, then its opposite face is gray? !5 The fact that the vast majority of people seem to systematically and predictably fail to deduce the correct answer to the Wason selection task is yet another challenge to classic notions of human rationality. This might also be an example of recruiting a heuristic instead of a more complex and demanding strategy to solve a reasoning task. At this point, I have only mentioned the negative story about human reasoning. Indeed, I have presented only the view that interprets these seeming errors in human reasoning as indications of irrationality. One might be surprised to know that there is a positive story to tell about these cases of human reasoning. Cue Gerd Gigerenzer (2008). Gigerenzer and colleagues have skillfully provided an account of how recruiting heuristical and biased reasoning strategies instead of more complex and thorough reasoning strategies might, overall, be a good idea. I imagine some readers will already have thought of a reason to rely on heuristics: they are more economical than the alternatives! And if there is a limited economy of cognitive energy, then recruiting cognitively efficient reasoning strategies is rational, right? This is roughly how Tversky and Kahneman (1974) and their ilk explain the use of heuristics and biases. This, however, is not Gigerenzer and colleagues' claim. With Henry Brighton (2009) and Wolfgang Gaissmaier (2011), Gigerenzer demonstrates that recruiting heuristics might be more mathematically rational, and not just more economically rational, then recruiting their alternatives. First, Gigerenzer and colleagues introduce their audience to a series of strategies, each of which is "biased" to varying degrees, that can be used to make estimations under conditions of uncertainty- e.g., the "take the best" strategy (Gigerenzer and Goldstein 1996) inspired by statistical models using "equal weights" or "tallying" strategies (Dawes 1974, 1979, Dawes and Corrigan 1974, Einhorn and Hogarth 1975, Schmidt 1971). Then Gigerenzer and colleagues show how the differential performance of these strategies can be modeled computationally, allowing for a quantitative adjudication between strategies. Comparing these estimation strategies across multiple data sets reveals that simpler-or more biased-strategies actually outperform models that are more thorough and sensitive to variance in the evidence-i.e., variance in the sample data set (Chater et al 2003, Goldstein and Gigerenzer 2002, Gigerenzer and Gaissmaier 2011, and Schooler and Hertwig 2005). If these differentially biased models of estimation strategies are taken to be analogous to certain !6 kinds of reasoning, then what Gigerenzer and Brighton show is that recruiting biased reasoning strategies-e.g., heuristics-might actually be more rational than recruiting unbiased strategies. Not coincidentally, these simulations seem to translate into ecologically valid human reasoning tasks (Hertwig and Todd 2003 and Gigerenzer 2007, 2008), meaning, simple-is-best or "less-is-more" cognitive strategies-where reasoners unconsciously estimate rather than fully calculate-are not only economically rational, but mathematically rational as well. While there are reasons to accept that claim, it is by no means uncontroversial or indefeasible. And it is important to note that even if one adopts Gigerenzer and colleagues' optimistic view of heuristics and biases, one still needs to face the fact that heuristics and biases inevitably result in some amount of error. In other words, even the optimistic view of biased reasoning fails to salvage the classic notion of perfect rationality. Furthermore, one can notice that heuristics will tend to outperform their more thorough and complex strategies only across multiple tasks or task-domains, meaning that it will not always be rational to rely on a heuristic. We began this section assuming that rationality entailed always and only complying with mathematical, logical, and/or economical norms. Then we discovered that this assumption contained problems. The work of Gigerenzer and colleagues has done well to modify this original perspective. The result is a sort of context-sensitive view of rationality (Gigerenzer and Hug 1992). According to this view, there will be circumstances in which being optimally rational entails complying with mathematical, logical, and/or economical norms, but there will also be circumstances in which being optimally rational requires recruiting heuristics and biases. This middle view-that the quality of a cognitive strategy is not absolute, but context-sensitive-serves as a handy segue into the topic of philosophical reasoning. 3. PHILOSOPHICAL REASONING Given that the rational reasoning strategy might be context-dependent, I can now pose the question of whether philosophy is a context in which logical, mathematical, and economical reasoning will turn out to be more rational or if it is a context in which heuristical and biased reasoning will be more rational. If one tried seriously to answer this question, I think !7 they would find that the dichotomy I have just suggested is a mistake. Philosophers seem to prize both of the reasoning strategies we have discussed so far, so it will not be so easy to claim that only one reasoning strategy will reign in the domain of philosophy. Having posed this question and mentioned a possible difficulty in providing a one-sided answer, I can begin to present further details about philosophy that will help guide the rest of this paper. There is some reason to think that philosophers could be called expert reasoners of some kind since philosophers outperform non-philosophers on various reasoning tasks. For example, philosophy majors outperform almost all other majors on the Graduate Record Examination (GRE) (see Educational Testing Service 2013) and the Law School Admission Test (LSAT) (see Nieswiadomy 2009, Table 1 and Table 2). Given that much of the LSAT is testing for competence in logical reasoning competence and multiple sections of the GRE are testing for mathematical competence, we might wonder if philosophy majors are, for whatever reason, better at quantitative reasoning. This would not be surprising in light of the fact most US philosophy departments require their students to take a course in formal logic-the mastery of which requires more than a modicum of quantitative competence (Prowse Tur and Thompson 2009). But quantitative competence does not fully capture the robust performance of philosophy majors since philosophy majors also outperform most of their peers in non-quantitative domains as well-e.g., the verbal and analytical writing section of the GRE. This apparent rhetorical or linguistic competency might also be attributable to education since philosophy majors are often graded, directly or indirectly, on the quality of their reading comprehension and argumentation. So perhaps the outstanding performance of philosophy majors on various non-quantitative reasoning tasks is also, at least in part, a result of their training. Consider also the fact that being a professional philosopher requires surviving multiple stages of highly competitive selection. For example graduate programs in philosophy often receive hundreds of applications and accept only a handful of students each year. Since only the top graduate students are admitted, and since the merits of each graduate student are either directly or indirectly related to various reasoning competencies, one might expect that reasoning performance would be saliently higher among students as a function of whether or not one has been admitted to a philosophy graduate program. Here I am making the case that !8 philosophical selection and training might account for the increase in certain reasoning competencies. To make this point more salient, I will briefly outline additional selection processes in academic philosophy. In graduate school students complete qualifying papers or qualifying exams to maintain their candidacy for a graduate degree and students also write and defend thesis or dissertation defenses in order to receive their degree. It would not be surprising if these processes also had a beneficial affect on philosophers' reasoning competence. And philosophers face further selection processes after graduate school in the way of hiring and promotion. It is unclear, however, whether these processes end up selecting for reasoning ability. It seems entirely possible that these processes merely select for something else, like the ability to make professional connections, to start new projects, to complete those projects, to meet deadlines, and to make a good impression, etc.- i.e., those abilities that result in more publications on one's curriculum vitae and better recommendations from one's colleagues. With some sense of how philosophers are trained and selected at the undergraduate, graduate, and professional level, I can go on to make some predictions. In addition to expecting professional philosophers to perform better on average on certain reasoning tasks than others, one would expect to find a positive correlation between reasoning ability and having or being a candidate for a PhD in philosophy and the number of years one has been studying philosophy. However, because hiring and promotion might not select for reasoning competence, one would not expect to find a correlation between reasoning ability and whether someone is employed to teach philosophy. One might also wonder about the gender distributions in professional philosophy. The profession of philosophy has become increasingly aware of the fact that philosophy departments tend to employ relatively few women (Buckwalter and Stich 2011). There are a variety of views about why this is the case, but there is some evidence that implicit bias at various stages in the undergraduate, graduate, and career-level training and selection processes could result in fewer opportunities for advancement for women (Paxton Figdor and Tiberius 2012, Stier and Yaish 2014). In an effort to counteract such biases, departments openly utilize affirmative action policies in their selection processes. Such policies usually require that when two candidates are equally qualified, and one candidate is underrep- !9 resented in philosophy, then the underrepresented individual will be chosen. If this is indeed how the affirmative action policies work, and if reasoning ability is among the dimensions on which two candidates could be equally qualified, then we should not expect to find large differences in reasoning performance among philosophers who have survived a selection process as a function of gender. There is at least one further prediction. It will require some explanation. The reader may remember that this section began with a comment about heuristics and biases on the one hand and the formally logical, mathematical, and/or economical reasoning strategies on the other. This came right after the suggestion that the use of one of these strategies over the other is not rational or irrational in itself. Instead, the rationality of using one of these strategies over the other was taken to depend on context. I also implied that philosophy might be a domain where both strategies are employed, depending on the context. Consider the nature of argumentation in analytic philosophy. Philosophers try to craft a series of premises that have a certain logical structure such that if the premises are true, then the conclusion must also be true. The ability to achieve this logical structure-and notice faults in the logical structure of arguments-is probably the most important tool in the analytic philosophers tool belt. But consider how difficult it can be to show that one's premises are true. After all, not all premises are subject to empirical investigation and conceptual analysis might not conclusively arbitrate the truth-value of a premise. Perhaps this is why philosophers will argue, explicitly or implicitly, that premises can be considered true or false in virtue of their intuitive appeal-viz., the premise just seems to be true or false (Audi 2004, Bealer 1998, Huemer 2005, Nagel 2007, 2013). There is little question that philosophers make these kinds of appeals (Audi 2004, Kornblith 1998, Talbot 2009, Chalmers 2013; for an opposing view, see Cappelen 2012). So, it seems that in addition to logical competence, analytic philosophers must also demonstrate some kind of competence in exercising intuition- including, perhaps, the intuition of their interlocutors. I take this difference between intuitive competence and logical competence to be loosely similar to the aforementioned difference between heuristic or biased reasoning strategies and logical, mathematical, or economical reasoning strategies. It will be helpful to simplify this distinction to a distinction between intuitive reasoning and reflective reason- !10 ing. For the remainder of the paper, 'intuitive' will refer to seemings or responses that are relatively effortless, possibly automatic, possibly associative, and prior to reflection. Conversely, 'reflective' will refer to judgments or responses that are relatively more effortful, calculative, and/or rule-based. I have already highlighted how philosophers' training might improve their reflective competence. One might go as far as to say that philosophers' training and selection improves their reflective competency more than it improves intuitive competency. Indeed, one might even concede that intuitive competency simply does not benefit from training and selection in philosophy. After all, it is not obvious how one would (intentionally) train one's intuitive competency or how intuitive competency would be selected for. This reveals the final expectation: that philosophers' training and selection will result in their being more likely than non-philosophers to overcome intuitive responses with reflective responses when the intuitive response is inferior to the reflective response. I will revisit this in the next two sections, so for now I can table further discussion of philosophical training and selection and how this might predict certain outcomes on reasoning tasks. For now, let me return to our discussion of the seeming dichotomy between two kinds of reasoning strategies. 4. DUAL-PROCESS THEORIES The aforementioned characterization of philosophical reasoning- i.e., the duality between intuitive reasoning and reflective reasoning-is by no means being doctored for the purposes of this paper. There is a resemblance to this dichotomy going back to ancient philosophy-'deliberation' vs. 'impulse' in Aristotle's Nichomachean Ethics Book III, 2-4 (Barnes 1984, 1111b4-1113b15) and Alexander of Aphrodisias's On Fate (Sharples 1983, 178.8-179.9)-as well as more recent philosophy-e.g., the salient differences between an intuitive response and a calculated utilitarian response that arise in certain thought experiments-e.g., the trolley cases (Hare 1981, 139-40), organ cases, etc. Perhaps more importantly, this dichotomy in reasoning strategies seems to have fallen quite naturally out of the empirical research on reasoning, learning, social cognition, and deci- !11 sion making (for a relatively brief, useful, and accessible introduction see Frankish 2010). Theories and models that endorse or assume the kind of dichotomy just described are often referred to as dual-process or dual-system theories (Chaiken and Trope 1999, Evans 2003, 2008, 2014a, 2014b, Evans and Stanovich 2013, Frankish 2010, Hammond 1996, Samuels 2009, Sloman 1996, 2002, Smith and DeCoster 2000, Thompson 2009, 2010, 2013, Wilson Lindsey and Schooler 2000). The duality referred to by 'dualprocess' has many names, each with it's own story: associative vs. rulebased (Sloman 1996), heuristic vs. analytic (Evans 1984, 1989), tacit thought vs. explicit thought (Evans and Over 1996), implicit cognition vs. explicit learning (Reber 1989), interactional vs. analytic (Levinson 1995), experiential vs. rational (Epstein 1994), quick and inflexive modules vs. intellection (Pollock 1991), intuitive cognition vs. analytical cognition (Hammond 1996), recognition primed choice vs. rational choice strategy (Klein 1998), implicit inference vs. explicit inference (Johnson-Laird 1983), automatic vs. controlled processing (Shiffrin and Schneider 1977), automatic activation vs. conscious processing system (Posner and Snyder 1975, 2004), rationality1 vs. rationality2 (Evans & Over 1996, 1997), intuitive vs. reflective (Frederick 2005), model-based vs. model-free (Daw et al 2005) and system 1 vs. system 2 (see Stanovich and West 2000 for the first mention of these terms as well as a useful, albeit dated, list of dualprocess terminology; see also Frankish 2010 for a list of features commonly associated with System 1 and System 2). While there are nuanced differences between certain dual-process theories, dual-process theories are those which claim that one can distinguish between at least two cognitive strategies in reasoning, learning, deciding, etc.-I emphasize that there might be more than two processes to avoid problems that result from positing "binary oppositions" (Newell 1973). One strategy is characterized by quick, effortless, and possibly associative seemings-referred to in this paper as intuitive-and the other of which is characterized by longer, more effortful, deliberative, perhaps calculative or even rule-based judgments- referred to in this paper as reflective. Although it is not entirely clear how these two strategies operate (e.g., serially vs. in parallel), how these strategies interact (e.g., competitively vs. non-competitively, mutually inhibitorily vs. complementarily, etc.), how these strategies are realized neurobiologically, or what these strategies actually are (e.g., systems, processes, styles, habits, personality traits, etc.), there are a variety of reasoning tasks !12 that demonstrate the dissociability of the two strategies (see Sloman 1996 for a helpful summary). The Cognitive Reflection Test In this paper, I will focus on a set of tasks that are designed to cue an intuitive response, which, upon reflection, can be shown to be false. These tasks are found in the Cognitive Reflection Test-henceforth the CRT (Frederick 2005). Take a look at an example of a CRT question (Figure 2); it could be useful to try to answer the question before continuing. ! The question is designed to elicit a particular intuitive response: 10 cents. This response is quick and effortless. It is also false. In order to explain why this is incorrect, consider the two criteria that need to be satisfied to correctly answer this question. Criterion 1: The cost of the bat and the ball must total $1.10. Criterion 2: The bat must cost $1.00 more than the ball. Notice that answering "10 cents" does not violate Criterion 1 by itself. After all, if the ball costs 10 cents, then Criterion 1 can be satisfied so long as the total cost comes to $1.10. So far, so good. A problem will arise, however, when we attempt to fulfill Criterion 2. We realize that if the ball costs 10 cents, then Criterion 2 requires that the bat cost $1.10, and when we add these two costs, the total is $1.20-which fails to satisfy Criterion 1. So if the ball costs 10 cents, then either Criteria 1 or Criterion 2 will not be satisfied. A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost? ______ cents Figure 2. Example of a CRT Question (Frederick 2005) !13 Demonstrating an analysis of the solution indicates how arriving at the correct answer would require a reflective strategy. Ideally, this reflective process would result in the correct answer: 5 cents. Shane Frederick (2005) created the CRT to demonstrate a certain kind of cognitive ability: namely the ability to overcome seemingly correct responses that turn out to be incorrect. But there is another use for the CRT. Since the CRT elicits specific intuitive responses and since the correct answer to a CRT question requires a reflective process, one can use CRT responses to distinguish between intuitive and reflective reasoning. That is, if a subject reports the quick and effortless response (e.g., 10 cents), then one can infer the use of intuitive reasoning. And if a subject reports the answer that requires deliberation and calculation (e.g., 5 cents), then one can infer the use of reflective reasoning-it might be worth noting that Frederick (2005) uses 'style' instead of 'reasoning.' One can also treat the CRT as a test of rationality in a specific domain. That is, the CRT can reveal a test-takers proclivity to respond intuitively vs. reflectively when it is rational to respond reflectively. There are a variety of interesting effects related to whether one answers intuitively or reflectively on the CRT. Historically, intuitive responses (e.g., 10 cents) were significantly related to being more willing to take risks to avoid losses than to achieve gains (Frederick 2005, Kahneman and Tversky 1979), reporting more confident belief in God-even after controlling for education level, income, political orientation, IQ, and personality, and various demographic variables-(Shenhav Rand and Greene 2012) reporting increased confidence in God's existence since childhood (ibid.), reporting belief in immortal souls (ibid.), and reporting experiences that convinced oneself of God's existence (ibid.). Similarly, reflective responses (e.g., 5 cents) were significantly related to being more patient (Frederick 2005), being more likely to choose larger-later rewards-as opposed to smaller-sooner rewards-(ibid.), being less sensitive to temporal discounting rates (ibid.), being more willing to gamble (ibid.), being less risk-seeking (ibid.), performing better in the American College Test (ACT) (ibid.), performing better on the Scholastic Aptitude Test (SAT) (ibid.), performing better on the Wonderlic Personnel Test (WPT) (ibid.), scoring higher on the Need For Cognition scale (NFC) (ibid.), having greater vocabulary IQ (Shenhav Rand and Greene 2012), demonstrating better matrix reasoning (ibid.), and demonstrating better probabilistic reasoning competence (Oechssler Roider and Schmitz 2009). !14 There have also been significant gender differences. After controlling for SATmath scores and CRT score itself, men answered reflectively significantly more than women-even though there are no significant gender differences in performance on the ACT, SATverbal, WPT, or NFC (Frederick 2005). Also, among those who answered incorrectly, women were more likely than men to respond intuitively- e.g., 10 cents (ibid.). Also, aforementioned correlations, like those between CRT performance and being more or less likely to choose larger-later (LL) rewards, were more salient for women than men (ibid.). Some of these findings-for example the finding that women were more likely then men to respond intuitively-are consistent with other dual-process task results (Aarnio and Lindeman 2005, Lieberman 2000, Pacini and Epstein 1999). There are also priming effects related to intuitive and reflective reasoning. For example, priming an intuitive reasoning strategy increases one's likelihood of reporting being convinced that God exists (Shenhav Rand and Greene 2012). This result aligns with other results in which intuitive thinking correlates positively with religious and paranormal beliefs (Aarnio and Lindeman 2005 Giannotti et al 2001, Pennycook et al 2012, Pennycook et al 2013, Pennycook [comments] 2013, Pennycook et al 2014a, 2014b, 2014c). Also of interest is the effect that priming the reflective strategy results in being more likely to make utilitarian moral judgments about a variety of scenarios (Paxton, Ungar, and Greene 2012). It is also worth noting that in a large study (n > 4000) philosophers were shown to be significantly more likely than non-philosophers to answer reflectively on the CRT (Livengood et al 2010): that is, training and selection in philosophy positively correlated with CRT performance. Characterizing The Intuitive-Reflective Dichotomy At this point, one might be assuming that the reflective strategy is always the best strategy. While this seems to be true in the case of the CRT, it is not clear how robust the superiority of reflection will be. There is some reason to think that there are limitations to the benefits of reflection. For example, there is evidence that thinking "too much" can actually lead to poorer judgments (Hertwig and Schooler 2003, Wilson and Schooler 1991). So, one should not necessarily conclude, at this point, that answering reflectively is indicative of a robustly optimal rational competence. It might be that answering reflectively is only the most rational strategy in certain domains-e.g., formal mathematical tasks, formal logical tasks, !15 etc. But even this view might be challenged by the work of Gigerenzer and colleagues-see Section 2. Also, I have been characterizing performance on the CRT in more than one way. In some instances, I characterize the dichotomy as the likelihood to recruit either the intuitive or the reflective reasoning, as if the two strategies are exclusive or mutually inhibitory. This is by no means the only way to characterize the differences in performance. It seems possible that intuition and reflection could function in tandem-a possibility that I accept. If so, then one might wonder whether and how intuition and reflection interact. One possibility is that reflection can double-check and possibly correct one's initial intuitive responses-this might be Frederick's (2005) view. This description assumes a certain ordering of the two strategies: namely, the intuitive strategy comes first-this seems to be consistent with Huemer's (2005, 101-103) view. Accepting this possibility might entail that all reasoners could register an intuitive response before having a chance to recruit the reflective strategy. And there is yet a third way to characterize the differences in performance on the CRT. In this characterization, one would grant the possibility that both intuitive and reflective reasoning can operate simultaneously. This allows for multiple characterizations of arbitration between intuitive and reflective responses. One possibility is that the two responses are in a winner-takes-all competition. On this characterization, answers to CRT questions would indicate which response won the competition. That is, if a reasoner answers intuitively, then one can infer that, for whatever reason, the intuitive response won out over the reflective response-and vice versa. Another possibility is that some independent process arbitrates between the two responses-cognitive scientists might hypothesize that part of the prefrontal cortex or the anterior cingulate cortex could play a role in this arbitration (Miller and Cohen 2001, Neys Vartanian and Goel 2008). A view that might be consistent with more than one of these characterizations is the view that people tend to be satisfied with their intuitive responses unless they sense some indication of difficulty about the task, in which case they might rely on a more reflective response (Alter et al 2007, 2013, Simmons and Nelson 2006, Thompson and Johnson 2014). It is too early to tell whether these results are generalizable to philosophers. One of the purposes in considering the various characterizations is to caution the reader from adopting one characterization without considering other possibilities (Livengood et al 2010). Another purpose is to take a !16 minute to appreciate some of the theoretical possibilities concerning the relationship between intuitive and reflective responses. Another purpose of presenting these various views is to admit that my final conclusion will commit me to some kind of arbitration between intuitive and reflective responses. I should be clear that while I assume this kind of arbitration, I by no means assume that the arbitration in consciously accessible (Neys Vartanian Goel 2008). I should also admit that I remain agnostic about which of the two aforementioned arbitration processes explains performance in the CRT. I have tried to outline the relationship between the intuitive and reflective responses. Now I will consider how to characterize each strategy individually. In the past, System 1-herein the intuitive strategy-has been described by cognitive scientists as quick (Kahneman 2011, Martignon and Hoffrage 2002, Thompson Evans and Campbell 2013) and associative (Sloman 1996, Talbot 2009) and System 2-herein the reflective strategy-has been described as rule-based (Sloman 1996), explicit (Evans and Over 1996, Reber 1989, Johnson-Laird 1983), controlled (Shiffrin and Schneider 1977), and conscious (Posner and Snyder 1975, 2004). I want to briefly distinguish my position on these descriptions. I will begin with the claim that intuitive responses will be quick. I think the claim is acceptable so long as by 'quick' one means relatively quick, where 'relative' refers to the individual. In other words, for any given reasoner, intuitive responses will probably become apparent relatively faster than reflective responses. However, this description does not entail that each reasoner's intuitive and reflective responses will become apparent within the same amount of time as another reasoner's responses. For instance, one reasoner, perhaps an expert, might have responded reflectively in the same amount of time that another reasoner, perhaps a novice, was only able to respond intuitively. This difference might seem unimportant, so I will briefly explain its significance. It is often assumed in cognitive science that relatively quick judgments and reaction times are the telltale signs of System 1, or intuitive reasoning. In other words if participants respond in a few seconds or as few as a few hundred milliseconds, then this reaction time is taken as sufficient evidence that the response was intuitive. Since this assumption might be faulty-as in the expert-novice example above-the practice of identifying intuitive responses, or System 1 responses in general, by reaction times (Gigerenzer and Goldstein 1996) might be subject to a kind of misattribution error. !17 Intuitive responses have also been characterized as, necessarily, associative (Talbot 2009). Now I will express a concern about this associationist conception of the intuitive strategy. The associationist about intuition will start by outlining a fairly well accepted tenet of cognitive science: whether or not we are aware of it, we are more or less constantly learning associations between things and properties of things via experience-e.g., if we frequently experience two things or properties of things together, then we will begin to associate them (Reber 1989, 1992, Talbot 2009, 28-48). The associationist will then point out that many associations are not consciously accessible and that one of the things consciously inaccessible associations do is influence consciously inaccessible judgments. The final claim is this: "intuitions are the conscious manifestation of [consciously inaccessible] judgments" (Talbot, 38). While parts of this view enjoy impressive empirical support and while this view offers good descriptions of some intuitions, I am not convinced that this view can capture everything that falls under the umbrella of 'intuition' in either cognitive science or philosophy. For example, it is rather unclear how an unconscious association would lead to an unconscious judgment that would lead to the response of "10 cents" on the aforementioned CRT question (Figure 2). Now I can turn to the descriptions of what I have called the reflective strategy. I will begin with 'ruled-based.' While I understand the motivation for this term-namely, that reflection and calculation might require some kind of logico-mathematico rule following process-it is not clear that this will be enough to distinguish the intuitive strategy from the reflective strategy. After all, the processes which underlie intuition might, at bottom, also instantiate rule following. In fact, even associative processes that are allegedly juxtaposed to rule-based processes (Sloman 1996) could be realized by a set of production rules (Eisenstadt and Simon 1997). The plausibility of this claim becomes clearer when one thinks about how both associative and rule-based cognitive processes can be modeled mathematically and computationally-that is, both can be instantiated by rule-following systems. I have a single concern with descriptions such as "explicit," "controlled," and "conscious." To make the concern apparent, consider a difference between expert and novice reasoning. Expert chess players are surely reasoning reflectively even if, unlike novices, much of their judgments and inferences are so well trained that they are consciously inacces- !18 sible. Indeed, they may not be able to exercise control over these judgments and might not be able to explicitly report on them during their reasoning. So if the requirements for reflective reasoning require explicit or conscious control, then expert reflection might be miscategorized as nonreflective. Meanwhile, philosophers do not agree on their notions of intuition (Kuntz and Kuntz 2011), perhaps because some are motivated by a favorable view of intuition (Audi 2004, Bealer 1998, Huemer 2005, Nagel 2007, 2013a, 2013b, Sosa 2006, 2007) and others are not (Swain Alexander Weinberg 2008, Weinberg 2007, Vaesen Peterson and Van Bezooijen 2013). For example George Bealer (1998) bends over backwards to distinguish 'intuition' from all empirically tractable concepts. His motivation for doing this, no doubt, is to arrive at a conclusion that allows for intuitions to count as evidence that can provide philosophers with uniquely powerful epistemic justifications in spite of damning empirical results. Michael Huemer has a similar motivation, but his definition of 'intuition' as "an intellectual appearance" or "the way things seem prior to reasoning" (2005, 101-102), while not being operationally defined, is more amenable to empirical investigation than Bealer's. Indeed, Huemer's definition of 'intuition' might even be consistent with what I have been calling intuitive in this paper. Since I am trying to cast a distinction between two reasoning styles in philosophy rather than weigh in on the debate of whether intuitions can count as evidence, we can set aside further discussion of what an intuition is and whether it can provide epistemological justification. Summary At this point it should be clear how intuitive and reflective responses fit into the landscape of dual-process theories and how performance on the CRT correlates with important differences between both reasoning ability and philosophical dispositions. The reader should also feel more familiar with characterizations of the dual-process dichotomy itself as well as each the various descriptions of each side of the dichotomy. More importantly, the reasons for avoiding certain descriptions and definitions of 'intuitive' or 'reflective' should be clear. With this background and clarification, I can move on to the next section, where I will consider a study of how philosophical training and selection correlates with performance on the CRT and how intuitive and reflective responses are differentially related to posi- !19 tions on central philosophical questions, regardless of one's training and selection in philosophy. 5. A CORRELATIONAL STUDY Introduction In Section 3, I mentioned that philosophy majors outperform most other majors on the LSAT and the GRE. In Section 4, I mentioned that philosophical training is positively correlated with performance on the CRT (Livengood et al 2010). This might be unsurprising given that reasoners with higher levels of analytic intelligence-perhaps those who have enjoyed philosophical training and selection-will tend to respond reflectively on various reasoning tasks (McCelroy and Seta 2003, Stanovich and West 2000). Naturally, then we should expect that having or being a candidate for a PhD in philosophy will positively correlate with answering reflectively on the CRT. The reader will also recall from Section 4 that intuitive responses on the CRT were significantly correlated with being convinced that God exists, with general belief in God, and with belief in immortal souls. Recall also that priming intuition resulted in reporting stronger belief in God and that priming reflection resulted in a greater likelihood of making utilitarian moral judgments. Based on these results, one might expect that people who have enjoyed philosophical training and selection will tend not to be theists and tend to make utilitarian judgments in certain scenarios. There were also gender effects in previous CRT data. Women were more likely than men to answer intuitively on the CRT. So one might wonder if this gender effect survives philosophical training and selection. Method A total of 562 subjects (111 female) from 36 countries-with more than 375 from the US-participated in a survey. The survey contained demographic questions, questions about ones position on central philosophical topics, and the CRT-in that order. Philosophers-undergraduates, graduates, and professionals-were recruited from LeiterReports.com and nonphilosophers were recruited from Amazon Mechanical Turk. In the section concerning one's philosophical position, the PhilPapers survey questions were used (Bourget et al 2009). Since many of the !20 PhilPapers questions require a familiarity with various topics in philosophy, the questions were adapted for some survey participants. At some point in the survey, subjects were asked if they had or were a candidate for a PhD in philosophy. Those who answered "yes" (n=318) received the same questions that appeared in the PhilPapers survey (2009). Those who answered "no" (n=244) to this question received more accessible adaptations of the questions (Figure 3). All but one of the philosophical questions required participants to choose between one of two positions. Similar to the format of the PhilPapers survey, participants indicated that they accepted one position, leaned towards one position, didn't know, or had no inclination (Figure 3). Participants were also presented with the CRT. The first question (see Figure 2) was adapted in order to be more amenable to those who are less familiar with bats and balls-'bat' and 'ball' were replaced with 'notebook' and 'pencil,' respectively. To control for familiarity with the CRT, subjects were asked whether they were familiar with the questions in !21 (if PhD in Philosophy) Free will: incompatibilism or compatibilism? __ Accept: incompatibilism __ Lean toward: incompatibilism __ I don't know __ No Inclination __ Lean toward: compatibilism __ Accept: compatibilism (if No PhD in Philosophy) If every event in the universe is determined, do you think it is possible that there could be free will? __ Accept: no __ Lean toward: no __ I don't know __ No Inclination __ Lean toward: yes __ Accept: yes Figure 3. Example of original and adapted PhilPapers question (Bourget et al 2009) the CRT and whether they knew any of the answers to the CRT before taking the survey. Following the design of Frederick 2005, the CRT instructions consisted in the following two sentences. Below are a series of math questions of varying difficulty. Please answer as many as you can. Subjects who did not consent, did not complete the survey, or did not take the survey seriously-e.g., one subject reported being a citizen of "Narnia"-were eliminated from the sample. All of the of the remaining data (n = 562) was included in all the following analysis. Results One measure of performance on the CRT is a CRT score (Frederick 2005, Paxton Ungar and Greene 2012), which is the number of questions answered correctly. Since there are three questions, 3.0 is the highest possible score. As predicted, having or being a candidate for a PhD in philosophy was positively correlated with CRT score (Cohen 1977). After controlling for previous familiarity with the CRT, those who had or were a candidate for a PhD in philosophy had a higher CRT score, on average, by 0.28; F(1, 558) = 15.41, p < 0.001, d = 0.32. Contrary to my predictions, the number of years spent studying philosophy, after accounting for whether or not one had or was a candidate for a PhD in philosophy, was not significantly related to one's CRT score, but as predicted, teaching philosophy was not statistically related to CRT score. Since the CRT score is an indication of how likely someone is to report a reflective answer, it might also be useful to have a metric of how likely someone is to report an intuitive answer. Let us call this an intuit score, which is the number of questions, out of three, that were answered with an intuitive response (e.g. 10 cents)-notice that incorrect responses are not necessarily intuitive responses. Similar to the last result, having or being a candidate for a PhD in philosophy was negatively correlated with one's intuit score. After controlling for previous familiarity with the CRT and sex, those who had or were a candidate for a PhD in philosophy had a lower intuit score, on average by 0.23; F(1, 557) = 14.72, p < 0.001, d = 0.31. And contrary to my predictions, the number of years spent studying philosophy, after accounting for whether or not one had or was a candidate for a PhD in philosophy, was not significantly related to one's intuit score -but as predicted, teaching philosophy was also not statistically related to CRT score. !22 Also, one's likelihood to respond intuitively was positively correlated with reporting theist inclinations. After controlling for whether or not one had or was a candidate for a PhD in philosophy, each intuitive response on the CRT was significantly related to being more likely to report either leaning toward or accepting theism; F(1, 559) = 7.3, p < 0.01, d = 0.16, b = 0.12. Responding intuitively was also was positively correlated with a Fregean view of language. Above and beyond the impact of one's familiarity with the CRT, every intuitive answer provided on the CRT was correlated with being more likely to report being inclined toward a Fregean, as opposed to a Russellian, view of language; F(1, 558) = 8.59, p < 0.01, d = 0.17, b = 0.19. There was also a marginally significant relationship between answering intuitively and one's view on scientific realism. For every intuitive response on the CRT, one was more likely to be an anti-realist about science; F(1, 560) = 4.26, p < 0.05, d = 0.12, b = 0.11- it should be noted that this result becomes statistically insignificant after controlling for familiarity with the CRT. Also as predicted, intuitive responses were correlated with less utilitarian responses to the trolley problem. After controlling for whether or not one had or was a candidate for a PhD in philosophy, intuitive responses on the CRT were negatively correlated with pulling the switch to save five and kill one; F(1, 559) = 6.93, p < 0.001, d = 0.15, b = 0.17. And as predicted, having or being a candidate for a PhD in philosophy is correlated with being significantly more likely to make a utilitarian judgment on the trolley problem. After controlling for one's likelihood of responding intuitively on the CRT, having or being a candidate for a PhD in philosophy is positively correlated with pulling the switch; F(1, 559) = 11.82, p < 0.001, d = 0.29, b = 0.32. Similarly, being prone to reflective responses on the CRT was also related to making utilitarian judgments on the trolley problem. For every reflective answer provided on the CRT, one was significantly more likely to pull the switch; F(1, 558) = 5.5, p < 0.05, d = 0.1, b = 0.13. Finally, answering intuitively was positively correlated with being a physicalist about personal identity. That is, after controlling for previous familiarity with the CRT, responding intuitively was associated with accepting the physical (as opposed to the psychological) view of personal identity; F(1, 558) = 8.57, p < 0.001, d = 0.17, b = 0.22. Furthermore, the present results were similar to extant gender effects. In this sample, one's sex (male or female) as well as one's gender was correlated with one's likelihood to answer reflectively or intuitively. !23 After controlling for familiarity with the CRT and whether one had or was a candidate for a PhD in philosophy, males' CRT scores were higher by 0.27-that is, males were more likely than females to answer reflectively; F(1, 557) = 8.95, p < 0.01, d = 0.30. In the same comparison, after accounting for sex and familiarity with the CRT, having or being a candidate for a PhD in philosophy increased one's CRT score by 0.29 F(1, 557) = 16.03, p < 0.0001, d = 0.33. Sex was also correlated with one's intuit score. After accounting for familiarity with the CRT and having or being a candidate for a PhD in philosophy, females intuit scores were higher by 0.26; F(1, 557) = 12.69, p < 0.001, d = 0.36. The magnitude of this correlation is only slightly larger than that of having or being a candidate for a PhD in philosophy in the same comparison. That is, after accounting for sex and familiarity with the CRT, having or being a candidate for a PhD decreased one's intuit score by 0.23; F(1, 557) = 14.72, p < 0.001, d = 0.31. Discussion Most of the aforementioned predictions were corroborated and all of the aforementioned results were reproduced. First, training and selection in philosophy resulted in better performance on the CRT-specifically, those who have been selected by PhD admissions committees in philosophy are more likely than those who have not been selected to answer reflectively on the CRT. Also, answering intuitively predicted a greater chance of accepting or leaning towards theism (as opposed to atheism), accepting a Fregean (as opposed to Russellian) view of language, accepting scientific anti-realism (as opposed to scientific realism) about science, choosing not pull the switch to save five and kill one, and to accept the physical (as opposed to psychological) view of personal identity. Conversely, answering reflectively predicted a greater chance of choosing to pull the switch to save five and kill one. And lastly, males were more likely to respond reflectively to CRT questions and females were more likely to respond intuitively, and, as predicted, the magnitude of these correlations with CRT score was similar to that of having or being a candidate for a PhD in philosophy. The reader should be aware that all of these correlations were small-defined as d ≈ 0.3 (Cohen 1977)-albeit statistically significant, so one would do well to take care in interpreting these results. In what remains of this section, I would like to point out some possibly confounding !24 factors of the present study, recommend some tentative conclusions, and then suggest directions for further research. First, I want to draw attention to the fact that the vast majority of participants came from LeiterReports.com. Since this website's audience is, primarily academic philosophers or aspiring academic philosophers, one might infer that the vast majority of participants in the present study probably had an academic interest in philosophy. And since training in philosophy-which is probably related to having an academic interest in philosophy-is related to the kind of reasoning ability that helps one perform well on the CRT, then we should expect this sample's CRT scores to be positively skewed-which seems apparent based on the comparison with Frederick's (2005) data and Livengood and colleagues' (2010) data (see Figure 4). This uneven distribution might also skew other results. For example, perhaps the correlation between having or being a candidate for PhD in philosophy and CRT score is diminished by the fact that the vast majority of those without a PhD in philosophy have nonetheless enjoyed substantial formal training and even selection by philosophy depart- !25 SAMPLE SCORE N PhD in Philosophy 2.35 238 Massachusetts Institute of Philosophy 2.18 61 No PhD in Philosophy 2.07 170 Princeton University 1.63 121 Boston Fireworks Display 1.53 195 Carnegie Mellon University 1.51 746 Harvard University 1.43 51 Some Grad-level Philosophical Training 1.32 158 University of Michigan: Ann Arbor 1.18 1267 Web-based Studies 1.10 525 Bowling Green University 0.87 52 University of Michigan: Dearborn 0.83 154 Michigan State University 0.79 118 University of Toledo 0.57 138 No Philosophical Training 0.44 823 Figure 4. Comparison of CRT performance. Bolded items denote data from this study. Italicized items denote data from Livengood et al 2010. All other items are from Frederick 2005. Most of Frederick's respondents were undergraduates from the universities listed. ments-e.g., those without a PhD in philosophy might still have an undergraduate degree in philosophy or a non-terminal graduate degree in philosophy, and/or they might have partially completed a PhD in philosophy. This PhD selection criterion was chosen because the PhD selection process is one of the earliest and most competitive selection processes of the various selection processes in philosophy. The idea, then, was that this criterion would do well to capture the effect of one kind of selection process in philosophy, if the effect existed. One might have reservations about using this criterion as opposed to, say, a criterion about whether or not one is employed as a philosophy instructor or professor. This seems like a reasonable reservation, however, the reader should recall that of those who had or were a candidate for a PhD in philosophy, neither (a) the number of years one had studied philosophy nor (b) whether one had a teaching position in philosophy were statistically related to CRT performance. This provides support for the claim in Section 3 that being employed or promoted in philosophy is not related to one's reasoning competence. One might also have reservations about whether the PhD criterion captures a distinction between "philosopher" and "non-philosopher." Personally, I take it that such a distinction-assuming it is realistic-will not be optimally captured by the PhD criterion. So, if one wanted to compare philosophers with non-philosophers, then one would do well to devise another criterion that does better to draw a line between participants that are familiar with philosophy and those that are probably not-e.g., US high school seniors vs. US philosophy majors (see also Livengood and colleagues' 2010 method of distinguishing levels of philosophical training). One might also wonder if an underrepresentation of females in the sample has skewed some of the data. It is not perfectly clear from the data which results would be most affected by this uneven distribution. Perhaps the correlations between sex or gender and CRT performance would be more salient or significant if the sample were more balanced. It is worth noting that the sex distribution in the present sample-roughly 20% female-is loosely representative of the recent data on sex distribution in professional philosophy-18.7% female in 2006 and 25.4% female in 2009 (Buckwalter and Stich 2011). Similarly, one might wonder if the present findings have anything to say about the underrepresentation of women in philosophy. Buckwalter and Stich (2011) argued that gender differences in intuition might help ex- !26 plain the relatively low numbers of women in professional philosophy. Adleberg, Thompson, and Nahmias (2014) tried to reproduce some of Buckwalter and Stich's results, but failed. One of their conclusions was this: if there are gender differences in philosophy, then they are not playing a role in driving women to leave or avoid philosophy. It is not clear that the findings presented herein provide any indication of a mechanism that would drive women to leave or avoid philosophy. It is also unclear how the present findings would support the hypothesis that there are gender differences in intuition among philosophers. If the present findings indicate anything about gender differences in philosophy, it is only that there is a small difference between males' and females' likelihood of reporting an intuitive vs. a reflective response in a domain where the reflective response should be preferred. But even this conclusion could be premature. It is true that, historically, male students outperformed female students in mathematics in high school-even though female students outperformed male students in elementary and middle school (Hyde Fennema and Lamon 1990). However, this performance gap was not robust; for instance, it was diminished when the sample included non-students (ibid.). Also, once females were taking as many advanced math and science courses as males and receiving as many undergraduate mathematics degrees as males, the gap became statistically insignificant (Hyde et al 2008)-a finding which has been shown more than once in both single-sex and coeducational settings (Pahlke Hyde and Mertz 2013). In other words the gender differences in math performance are not the result of large and essential gender differences; rather, they are the result of differences in education. But perhaps other factors also play a role in gender differences in math performance. For example cross-national studies show that "gender equity in school enrollment, women's share of research jobs, and women's [governmental] representation" in each nation are the most powerful predictors of variance in math performance by gender (Else-Quest Hyde and Linn 2010). These results, as well as the results herein, corroborate the gender similarities hypothesis: "most psychological gender differences are in the close-tozero (d ≤ 0.10) or small (0.11 < d < 0.35) range, a few are in the moderate range (0.36 < d < 0.65), and very few are large (d < 0.66–1.00) or very large (d < 1.00)" (Hyde 2005). The implication is that gender differences in math performance are related to educational and cultural differences rather than essential gender differences. So in the case of the present study, !27 one has reason to be suspect of the claim that the present sex difference in performance on the CRT is an indication of some kind of essential sex difference in mathematical competence. The final worry about these findings involves dominant views in philosophy. Consider the fact that of those who reported an inclination about the existence of a god, 17% reported leaning toward atheism and 63% reported accepting atheism-meaning 80% were inclined toward atheism. This is clearly an uneven distribution, so one might be curious how it affects the data. One can be optimistic about the distribution's affect on statistical power since the present sample size is sufficiently large to contain over 100 participants who report an inclination towards theism. Also, the distribution in the present sample is remarkably close to the 78% found in the more comprehensive PhilPapers sample (Bourget et al 2009), so one should not be concerned about whether or not the present sample is representative of professional philosophers. There is yet another curiosity about this distribution. One might worry that the quantitative dominance of atheists is the result of some kind of indoctrination effect during the training and/or selection of philosophers (Cronk God Is Not Dead). After all, philosophical training and selection is related to philosophers' outstanding reflective competence and philosophers' outstanding reflective competence is related to a disposition towards atheism, so this worry seems prima facie reasonable. However, there are reasons to dispel this worry. First, having or being a candidate for a PhD in philosophy was not significantly related to one's inclinations about the existence of a god. Second, 70% of philosophers who report philosophy of religion as an area of specialization are theists (De Cruz 2012). In other words, philosophical training or selection does not seem to be related to philosophers' views about the existence of a god; it is philosophers' interests or specializations that seem to be so related. One might wonder if similar analyses of other dominant views in philosophy (e.g., moral realism) would dismiss other misconceptions of philosophy. Before turning to conclusions, I want to briefly mention some important differences between the present study and Livengood and colleagues' study (2010). The most obvious difference is that the present study considers correlations between CRT performance and philosophical disposition. In addition to this difference, the present study seems to indicate significantly higher performance, on average, than the Livengood et al study. It might be that having relatively fewer non-philosopher and female !28 participants in the present study could account for some of this difference. Another factor that might account for this difference is the fact that Livengood and colleagues controlled for overall education and age while I did not. Another interesting difference between Livengood and colleagues' study and the present study is the effect of philosophical training on CRT performance. Livengood and colleagues report that the average CRT score of participants with graduate-level philosophical training-Ŷ = 1.32-was about three times greater than the average CRT score of participants without philosophical training-Ŷ = 0.44. This seemingly massive disparity would not be possible without the alarmingly low average CRT score of subjects without philosophical training. Another possible cause of this large mean difference is the fact that Livengood and colleagues' might not have controlled for participants' previous familiarity with the CRT questions or answers. In the present study, familiarity with CRT questions turned out to be the most powerful predictor of both CRT score-F(1, 557) = 10.58, p < 0.01, d = 0.37, b = 0.33-and intuit score-F(1, 557) = 12.52, p < 0.001, d = 0.4, b = -0.29-even after accounting for one's sex and whether one had or was a candidate for a PhD in philosophy. So, one might wonder whether such a control would diminish the relatively larger differences in CRT score reported by Livengood and colleagues. Conclusions Let us now consider some tentative conclusions. First, the present results are not sufficiently robust to draw sweeping conclusions about the overall rationality of various groups-e.g., PhDs in philosophy vs. others, males vs. females, etc. The reason for this has been mentioned more than once: rationality might be domain-dependent. So, the rational strategy on the CRT, to respond reflectively, might not be the best strategy in another domain or across multiple domains (Gigerenzer and Brighton 2009, Gigerenzer and Gaissmaier 2011). Second, the present study does corroborate the prediction that training and selection in philosophy is related to an increase in one kind of reasoning competence-i.e., competency in responding reflectively rather than intuitively when it is rational to do so. Interestingly, the present results indicate that being hired or promoted in philosophy is not related to this kind of reasoning competence. Third, the present study corroborates the prediction that differences in reasoning competence are related to philosophical beliefs, even among those who have received the benefits of philosophical training and selection. This corroboration is !29 the most tenuous, however, because the reported correlations are rather small. Future Research Having reviewed the tentative conclusions and some constraining factors, I have only to mention some suggestions for further research. First, I suggest-following Shenhav Rand and Greene 2012-that the potential effects of intuitive or reflective priming on philosophers' responses to philosophical questions and scenarios be investigated. Specifically, I propose a 2x2 between subjects design, in which participants are randomly assigned to an intuition-eliciting, an intuition-inhibiting, a reflection-eliciting, or a reflection-inhibiting prime followed by a manipulation check. After this, subjects would be presented with whatever material would serve as the dependent variable: questions either about one's position on major philosophical views or about scenarios (e.g., thought experiments about identity, causality, mindedness, responsibility, etc.). One might also turn the proposed study into a 2x2x2 design by testing the difference between another pair of groups: philosophers and non-philosophers. It might even be useful to split philosophers into multiple groups-e.g., undergraduates, graduate students, non-tenure track faculty, tenure-track faculty, tenured faculty, and emeritus faculty-to see if there are any interesting differences between these groups. And since certain personality traits have been shown to be related to political orientation (Arvan 2011, 2012, Mondak and Conache 2014), views about moral responsibility (Schulz Cokely and Feltz 2011), and reasoning style (Messick and Fritzky 1963, Paccini and Epstein 1999), I want to propose studies of the potential relationships between personality traits, reasoning styles, and philosophical dispositions. The study could involve a brief presentation of the dependent variable-again, questions either about one's position on major philosophical views or about scenarios-and then proceed to present a Big Five personality assessment (Gosling et al 2003) and the CRT (Frederick 2005). Before closing, I want to mention a more ambitious proposal for further research. There is a growing body of research showing that differences in brain anatomy and function correlate with differences in responses to, judgments about, and orientation in particular areas of philosophy. For example, it seems that differences in brain anatomy and/or function are related to differences in political orientation (Amodio et al 2007, Kanai !30 et al 2011, Koenigs et al 2007, Tusche et al 2013), differences in judgments about mindedness (Jack et al 2012, 2013a, 2013b, 2014), differences in moral judgment (Moll and Oliviera-Souza 2007, Moll et al 2005, Young et al 2007, 2010a, 2010b, 2011), and differences in reasoning about religion (Kapogiannis et al 2009). This research has done well to reveal how various reasoning systems are related-e.g., moral reasoning and reasoning about other minds (Gray Young and Waytz 2012)-in lay reasoners. Also there has been inspiring research on how the brain realizes various kinds of reasoning (Goel 2007, Goel and Dolan 2001, 2003a, 2003b, 2004, Goel and Grafman 2000, Goel and Vartanian 2005, Goel et al 1998, 2000). By adding groups of philosophers to studies in the cognitive neuroscience of reasoning, one might achieve greater clarity on whether or not -and how-judgments and intuitions differ both between philosophers and non-philosophers and within philosophers. 6. CONCLUSION This paper began with a brief discussion of how cognitive science has provided reasons to doubt traditional conceptions of human rationality. This discussion of what counts as rational and whether or not humans are optimally rational is clearly not settled. But out of the same research from which these concerns about rationality were borne came dual-process accounts of reasoning, learning, deciding, etc. These dual-process accounts of reasoning have helped to characterize the intuitive-reflective dichotomy found in philosophical reasoning. Also, the CRT helped to reveal that philosophers are outstandingly competent in arbitrating between reflective and intuitive responses such that philosophers are more likely to respond reflectively when it is rational to do so. The fact that training and selection in philosophy is negatively correlated with making such systematic errors seems to indicate that some of the worries about the use of intuition in philosophy-e.g., that responding intuitively is irrational because it can lead to systematic errors on the CRT-can be mitigated. Still, the present findings also suggest that training or selection in philosophy does not completely eliminate the proclivity for systematic errors in arbitrating between intuitive and reflective responses. My conclusion, then, involves ascribing a mitigated privilege to philosophers' ability to rationally arbitrate between intuitive and reflective responses. After all, !31 the present study provides at least prima facie reasons to think that philosophical training and selection result in being more likely to rationally arbitrate between intuitive and reflective responses. Nonetheless, the present study also provides prima facie reason to think that, as a rule, "intuitions should not be embraced [unreflectively]" (Huemer 2005, 105). There is one important disclaimer I should make before wrapping up: I have not suggested that philosophical training and selection might improve the quality of one's intuitions (Knobe and Samuels 2013, Schwitzgebel and Cushman 2012, Stotz et al forthcoming, Tobia Buckwalter and Stich 2012, Tobia Chapman and Stich 2013, Weinberg et al 2010, Williamson 2011). Instead, I am claiming that philosophical training and/ or selection might improve one's ability to rationally adjudicate between intuition and reflection. The reader will perhaps be surprised or disappointed that I have not arrived at a conclusion about when intuitions should count as evidence, when intuitions should ground non-empirical claims, or when appealing to intuitions is rational. I confess that I share a portion of that surprise and disappointment. If I am honest, I began thinking about philosophical reasoning a few years ago with the hope of providing both empirical evidence and a knockdown argument against the method of appealing to intuition in philosophy. While some of my research in cognitive science and philosophy has supported my suspicions about appealing to intuitions, it has also eroded my confidence in full-blown "intuition-bashing" projects (Bealer 1998). !32 BIBLIOGRAPHY Aarnio, Kia, and Marjaana Lindeman. "Paranormal Beliefs, Education, and Thinking Styles." Personality and Individual Differences 39, no. 7 (2005): 1227–36. Ackerman, Joshua M., Jenessa R. Shapiro, and Jon K. Maner. "When Is It Good to Believe Bad Things?," 2010. Adleberg, Toni, Morgan Thompson, and Eddy Nahmias. "Do Men and Women Have Different Philosophical Intuitions? Further Data." Philosophical Psychology, forthcoming, 1–27. Alexander, Joshua, and Jonathan M. Weinberg. "Analytic Epistemology and Experimental Philosophy." Philosophy Compass 2, no. 1 (January 1, 2007): 56–80. doi:10.1111/j.1747-9991.2006.00048.x. Alfano, Mark, and Don Loeb. "Experimental Moral Philosophy." In The Stanford Encyclopedia of Philosophy, edited by Edward N. Zalta, Spring 2014., 2014. http://plato.stanford.edu/archives/spr2014/entries/experimental-moral/. Alter, Adam L., Daniel M. Oppenheimer, and Nicholas Epley. "Disfluency Prompts Analytic thinking-But Not Always Greater Accuracy: Response to." Cognition 128, no. 2 (August 2013): 252–55. doi: 10.1016/j.cognition.2013.01.006. Alter, Adam L., Daniel M. Oppenheimer, Nicholas Epley, and Rebecca N. Eyre. "Overcoming Intuition: Metacognitive Difficulty Activates Analytic Reasoning." Journal of Experimental Psychology: General 136, no. 4 (2007): 569. Amodio, David M., John T. Jost, Sarah L. Master, and Cindy M. Yee. "Neurocognitive Correlates of Liberalism and Conservatism." Nature Neuroscience 10, no. 10 (2007): 1246–47. Aristotle. The Complete Works of Aristotle: The Revised Oxford Translation, Vol. 2. Edited by Jonathan Barnes. Princeton, N.J.: Princeton University Press, 1984. Arvan, Marcus. "A Lot More Bad News for Conservatives, and a Little Bit of Bad News for Liberals? Moral Judgments and the Dark Triad Personality Traits: A Follow-up Study." Neuroethics, 2012, 1–14. ---. "Bad News for Conservatives? Moral Judgments and the Dark Triad Personality Traits: A Correlational Study." Neuroethics, 2011, 1–12. Ball, Linden J., Peter Phillips, Caroline N. Wade, and Jeremy D. Quayle. "Effects of Belief and Logic on Syllogistic Reasoning." Experimental Psychology (formerly "Zeitschrift Für Experimentelle Psy- !33 chologie") 53, no. 1 (January 1, 2006): 77–86. doi: 10.1027/1618-3169.53.1.77. Ball, Linden J., E. J. Sutherland, and J. D. Quayle. "Situating Deductive Competence Within A Models Framework: Some Further Considerations." Cahiers de Psychologie Cognitive 16, no. 1–2 (n.d.): 52– 62. Accessed March 29, 2014. Bar-Hillel, Maya. "On the Subjective Probability of Compound Events." Organizational Behavior and Human Performance 9, no. 3 (June 1973): 396–406. doi:10.1016/0030-5073(73)90061-5. ---. "Representativeness and Fallacies of Probability Judgment." Acta Psychologica 55, no. 2 (1984): 91–107. ---. "Similarity and Probability." Organizational Behavior and Human Performance 11, no. 2 (1974): 277–82. ---. "The Base-Rate Fallacy in Probability Judgments." Acta Psychologica 44, no. 3 (1980): 211–33. Bara, Bruno G, Monica Bucciarelli, and Philip N Johnson-Laird. "Development of Syllogistic Reasoning." American Journal of Psychology 108 (1995): 157–157. Barwise, Jon. "Everyday Reasoning and Logical Inference." Behavioral and Brain Sciences 16, no. 02 (1993): 337–38. doi:10.1017/ S0140525X00030314. Bealer, George. "Intuition and the Autonomy of Philosophy." edited by Michael Depaul and William Ramsey, 201–39. Rethinking Intuition: The Psychology of Intuition and Its Role in Philosophical Inquiry. Lanham: Rowman & Littlefield, 1998. ---. "On the Possibility of Philosophical Knowledge." Noûs 30 (January 1, 1996): 1–34. doi:10.2307/2216234. Bell, Victoria A., and P. N. Johnson-Laird. "A Model Theory of Modal Reasoning." Cognitive Science 22, no. 1 (January 1998): 25–51. doi:10.1016/S0364-0213(99)80034-2. Bem, Daryl J. "An Experimental Analysis of Self-Persuasion." Journal of Experimental Social Psychology 1, no. 3 (1965): 199–218. ---. "Self-Perception Theory." In Advances in Experimental Social Psychology, Vol. 8. New York: Academic Press, 1978. Bem, Daryl J., and H. Keith. "Testing the Self-Perception Explanation of Dissonance Phenomena: On the Salience of Premanipulation Attitudes." Journal of Personality and Social Psychology 14, no. 1 (1970): 23–31. doi:10.1037/h0020916. Bem, Sacha. "The Explanatory Autonomy of Psychology Why a Mind Is Not a Brain." Theory & Psychology 11, no. 6 (2001): 785–95. !34 Bendixen, Lisa D., Gregory Schraw, and Michael E. Dunkle. "Epistemic Beliefs and Moral Reasoning." The Journal of Psychology 132, no. 2 (1998): 187–200. doi:10.1080/00223989809599158. Bengson, John. "Experimental Attacks on Intuitions and Answers." Philosophy and Phenomenological Research 86, no. 3 (May 1, 2013): 495–532. doi:10.1111/j.1933-1592.2012.00578.x. Bonatti, Luca. "Propositional Reasoning by Model?" Psychological Review 101, no. 4 (1994): 725–33. doi:10.1037/0033-295X. 101.4.725. Bourget, David, and David Chalmers. "The PhilPapers Surveys," 2009. http://philpapers.org/surveys/. ---. "What Do Philosophers Believe?" Philosophical Studies, n.d., 1– 36. Accessed March 6, 2014. doi:10.1007/s11098-013-0259-7. Braine, Martin DS. "Steps toward a Mental Predicate Logic." Mental Logic, 1998, 273–331. Braine, Martin DS, and David P O'Brien. Mental Logic. Psychology Press, 1998. Brewka, Gerhard. Nonmonotonic Reasoning: Logical Foundations of Commonsense. Vol. 12. Cambridge University Press, 1991. Buckwalter, Wesley. "Non-Traditional Factors in Judgments about Knowledge." Philosophy Compass 7, no. 4 (April 1, 2012): 278–89. doi: 10.1111/j.1747-9991.2011.00466.x. Buckwalter, Wesley, and Stephen Stich. "Gender and Philosophical Intuition." Experimental Philosophy 2 (2010). Byrne, Ruth M.J. "Suppressing Valid Inferences with Conditionals." Cognition 31, no. 1 (February 1989): 61–83. doi: 10.1016/0010-0277(89)90018-8. Byrne, Ruth M.J., and Simon J Handley. "Reasoning Strategies for Suppositional Deductions." Cognition 62, no. 1 (February 2, 1997): 1–49. doi:10.1016/S0010-0277(96)00720-2. Byrne, Ruth M.J., Simon J. Handley, and Philip N. Johnson-Laird. "Reasoning from Suppositions." The Quarterly Journal of Experimental Psychology Section A 48, no. 4 (1995): 915–44. doi: 10.1080/14640749508401423. Cappelen, Herman. Philosophy without Intuitions. 1st ed. Oxford: Oxford University Press, 2012. Chaiken, Shelly, and Yaacov Trope. Dual-Process Theories in Social Psychology. Guilford Press, 1999. !35 Chalmers, David J. "Intuitions in Philosophy: A Minimal Defense." Forthcoming in Philosophical Studies, 2013. Chan, David, and Fookkee Chua. "Suppression of Valid Inferences: Syntactic Views, Mental Models, and Relative Salience." Cognition 53, no. 3 (December 1994): 217–38. doi: 10.1016/0010-0277(94)90049-3. Chapman, Loren, and Jean Chapman. "Genesis of Popular But Erroneous Psychodiagnostic Observations." Journal of Abnormal Psychology 72, no. 3 (1967): 193–204. doi:10.1037/h0024670. Chase, Valerie M, Ralph Hertwig, and Gerd Gigerenzer. "Visions of Rationality." Trends in Cognitive Sciences 2, no. 6 (June 1, 1998): 206–14. doi:10.1016/S1364-6613(98)01179-6. Chater, Nick, and Mike Oaksford. "Human Rationality and the Psychology of Reasoning: Where Do We Go from Here?" British Journal of Psychology 92, no. 1 (February 1, 2001): 193–216. doi: 10.1348/000712601162031. ---. "Logicism, Mental Models and Everyday Reasoning: Reply to Garnham." Mind & Language 8, no. 1 (March 1, 1993): 72–89. doi:10.1111/j.1468-0017.1993.tb00271.x. Chater, Nick, Mike Oaksford, Ramin Nakisa, and Martin Redington. "Fast, Frugal, and Rational: How Rational Norms Explain Behavior." Organizational Behavior and Human Decision Processes 90, no. 1 (January 2003): 63–86. doi:10.1016/ S0749-5978(02)00508-3. Cheng, Patricia W, and Keith J Holyoak. "Pragmatic Reasoning Schemas." Cognitive Psychology 17, no. 4 (October 1985): 391–416. doi: 10.1016/0010-0285(85)90014-3. Chi, Michelene T. H., Paul J. Feltovich, and Robert Glaser. "Categorization and Representation of Physics Problems by Experts and Novices." Cognitive Science 5, no. 2 (1981): 121–52. doi:10.1207/ s15516709cog0502_2. Chin, Simone L, and Alan W. Kersten. "The Application of the Less Is More Hypothesis in Foreign Language Learning," 2009. Cohen, Jacob. Statistical Power Analysis For Behavioral Sciences. Revised Edition. New York: Academic Press, 1977. Cohen, L. Jonathan. "Can Human Irrationality Be Experimentally Demonstrated?" Behavioral and Brain Sciences 4, no. 03 (1981): 317–31. doi:10.1017/S0140525X00009092. Cronk, Harold. God Is Not Dead. Drama. Pure Flix Entertainment, Freestyle Releasing, 2014. !36 Cummins, Denise D., Todd Lubart, Olaf Alksnis, and Robert Rist. "Conditional Reasoning and Causation." Memory & Cognition 19, no. 3 (May 1, 1991): 274–82. doi:10.3758/BF03211151. Damasio, Antonio R. "Descartes' Error: Emotion, Rationality and the Human Brain." New York: Putnam 352 (1994). Daw, Nathaniel D., Samuel J. Gershman, Ben Seymour, Peter Dayan, and Raymond J. Dolan. "Model-Based Influences on Humans' Choices and Striatal Prediction Errors." Neuron 69, no. 6 (March 24, 2011): 1204–15. doi:10.1016/j.neuron.2011.02.027. Dawes, Robyn M. "The Robust Beauty of Improper Linear Models in Decision Making." American Psychologist 34, no. 7 (1979): 571–82. doi:10.1037/0003-066X.34.7.571. Dawes, Robyn M., and Bernard Corrigan. "Linear Models in Decision Making." Psychological Bulletin 81, no. 2 (1974): 95–106. doi: 10.1037/h0037613. De Cruz, Helen. "Preliminary Results Of The Survey On Natural Theological Arguments," 2012. https://www.academia.edu/1438058/Results_of_my_survey_on_natural_theological_arguments. Dugan, Charles M., and Russell Revlin. "Response Options and Presentation Format as Contributors to Conditional Reasoning." The Quarterly Journal of Experimental Psychology Section A 42, no. 4 (1990): 829–48. doi:10.1080/14640749008401252. Dunaway, Billy, Anna Edmonds, and David Manley. "The Folk Probably Do Think What You Think They Think." Australasian Journal of Philosophy 91, no. 3 (2013): 421–41. doi: 10.1080/00048402.2012.724430. Edwards, W., L.D. Phillips, William L. Hays, and B.C. Goodman. "Probabilistic Information Processing Systems: Design and Evaluation." IEEE Transactions on Systems Science and Cybernetics 4, no. 3 (September 1968): 248–65. doi:10.1109/TSSC.1968.300119. Einhorn, Hillel J., and Robin M. Hogarth. "Unit Weighting Schemes for Decision Making." Organizational Behavior and Human Performance 13, no. 2 (April 1975): 171–92. doi: 10.1016/0030-5073(75)90044-6. Eisenstadt, Stuart A., and Herbert A. Simon. "Logic and Thought." Minds and Machines 7, no. 3 (August 1, 1997): 365–85. doi:10.1023/A: 1008299628430. Elio, Renée, ed. Common Sense, Reasoning, & Rationality. New York: Oxford University Press, 2002. !37 Else-Quest, Nicole M., Janet Shibley Hyde, and Marcia C. Linn. "CrossNational Patterns of Gender Differences in Mathematics: A MetaAnalysis." Psychological Bulletin 136, no. 1 (2010): 103–27. doi: 10.1037/a0018053. Epley, Nicholas, and Thomas Gilovich. "Putting Adjustment Back in the Anchoring and Adjustment Heuristic: Differential Processing of Self-Generated and Experimenter-Provided Anchors." Psychological Science 12, no. 5 (2001): 391–96. ---. "The Anchoring-and-Adjustment Heuristic Why the Adjustments Are Insufficient." Psychological Science 17, no. 4 (2006): 311–18. Epley, Nicholas, Boaz Keysar, Leaf Van Boven, and Thomas Gilovich. "Perspective Taking as Egocentric Anchoring and Adjustment." Journal of Personality and Social Psychology 87, no. 3 (2004): 327. Epstein, Seymour. "Integration of the Cognitive and the Psychodynamic Unconscious." American Psychologist 49, no. 8 (1994): 709. Evans, Jonathan. Bias in Human Reasoning: Causes and Consequences. Vol. ix. Essays in Cognitive Psychology. Hillsdale, NJ, England: Lawrence Erlbaum Associates, Inc, 1989. ---. Bias in Human Reasoning: Causes and Consequences. London: Erlbaum, 1989. ---. "Dual-Processing Accounts of Reasoning, Judgment, and Social Cognition." Annu.Rev.Psychol. 59 (2008): 255–78. ---. "Heuristic and Analytic Processes in Reasoning*." British Journal of Psychology 75, no. 4 (November 1, 1984): 451–68. doi: 10.1111/j.2044-8295.1984.tb01915.x. ---. "In Two Minds: Dual-Process Accounts of Reasoning." Trends in Cognitive Sciences 7, no. 10 (2003): 454–59. ---. "Logic and Human Reasoning: An Assessment of the Deduction Paradigm." Psychological Bulletin 128, no. 6 (2002): 978–96. doi: 10.1037/0033-2909.128.6.978. ---. "Matching Bias in Conditional Reasoning: Do We Understand It After 25 Years?" Thinking & Reasoning 4, no. 1 (1998): 45–110. doi:10.1080/135467898394247. ---. Reasoning, Rationality and Dual Processes: Selected Works of Jonathan Evans, 2014. ---. "The Influence of Linguistic Form on Reasoning: The Case of Matching Bias." The Quarterly Journal of Experimental Psychology Section A 52, no. 1 (1999): 185–216. doi:10.1080/713755805. !38 ---. "The Mental Model Theory of Conditional Reasoning: Critical Appraisal and Revision." Cognition 48, no. 1 (July 1993): 1–20. doi:10.1016/0010-0277(93)90056-2. ---. "Two Minds Rationality." Thinking & Reasoning 20, no. 2 (2014): 129–46. doi:10.1080/13546783.2013.845605. Evans, Jonathan, John Clibbens, and Benjamin Rood. "The Role of Implicit and Explicit Negation in Conditional Reasoning Bias." Journal of Memory and Language 35, no. 3 (June 1996): 392–409. doi: 10.1006/jmla.1996.0022. Evans, Jonathan, and A. E. Dusoir. "Proportionality and Sample Size as Factors in Intuitive Statistical Judgement." Acta Psychologica 41, no. 2 (1977): 129–37. Evans, Jonathan, Newstead, and Ruth M. J Byrne. Human Reasoning: The Psychology of Deduction. Hove: Lawrence Erlbaum Associates, 1993. Evans, Jonathan, Stephen E. Newstead, and Ruth M. J. Byrne. Human Reasoning: The Psychology of Deduction. Psychology Press, 1993. Evans, Jonathan, and David E Over. "Rationality in Reasoning: The Problem of Deductive Competence." Current Psychology of Cognition 16, no. 1–2 (n.d.): 3–38. Evans, Jonathan, and David E. Over. "Rationality in the Selection Task: Epistemic Utility versus Uncertainty Reduction." Psychological Review 103, no. 2 (1996): 356–63. doi:10.1037/0033-295X. 103.2.356. Evans, Jonathan, and Keith E. Stanovich. "Dual-Process Theories of Higher Cognition Advancing the Debate." Perspectives on Psychological Science 8, no. 3 (May 1, 2013): 223–41. doi: 10.1177/1745691612460685. Falmagne, Rachel J., and Joanna Gonsalves. "Deductive Inference." Annual Review of Psychology 46, no. 1 (1995): 525–59. doi:10.1146/ annurev.ps.46.020195.002521. Federico, Christopher M., Christopher R. Weber, Damla Ergun, and Corrie Hunt. "Mapping the Connections between Politics and Morality: The Multiple Sociopolitical Orientations Involved in Moral Intuition." Political Psychology, 2013. Feltz, Adam, and Edward T. Cokely. "The Philosophical Personality Argument." Philosophical Studies 161, no. 2 (November 1, 2012): 227–46. doi:10.1007/s11098-011-9731-4. Finetti, Bruno de. "The Value of Studying Subjective Evaluations of Probability." In The Concept of Probability in Psychological Experi- !39 ments, edited by Carl-Axel S. Staël Von Holstein, 1–14. Theory and Decision Library 8. Springer Netherlands, 1974. http:// link.springer.com/chapter/10.1007/978-94-010-2288-0_1. Fiske, Alan Page, and Philip E. Tetlock. "Taboo Trade-Offs: Reactions to Transactions That Transgress the Spheres of Justice." Political Psychology 18, no. 2 (1997): 255–97. doi:10.1111/0162-895X. 00058. Ford, Marilyn. "Two Modes of Mental Representation and Problem Solution in Syllogistic Reasoning." Cognition 54, no. 1 (January 1995): 1–71. doi:10.1016/0010-0277(94)00625-U. Forsyth, Donelson R. "The Functions of Attributions." Social Psychology Quarterly 43, no. 2 (June 1, 1980): 184–89. doi:10.2307/3033621. Frankish, Keith. "Dual-Process and Dual-System Theories of Reasoning." Philosophy Compass 5, no. 10 (October 1, 2010): 914–26. doi: 10.1111/j.1747-9991.2010.00330.x. Frederick, Shane. "Automated Choice Heuristics." In Heuristics and Biases: The Psychology of Intuitive Judgment, edited by T. Gilovich, D. Griffin, and D. Kahneman, 548–58. New York, NY, US: Cambridge University Press, 2002. ---. "Cognitive Reflection and Decision Making." Journal of Economic Perspectives 19, no. 4 (December 2005): 25–42. doi: 10.1257/089533005775196732. Frost, Laurie A., Janet S. Hyde, and Elizabeth Fennema. "Chapter 2 Gender, Mathematics Performance, and Mathematics-Related Attitudes and Affect: A Meta-Analytic Synthesis." International Journal of Educational Research 21, no. 4 (1994): 373–85. doi:10.1016/ S0883-0355(06)80026-1. Galbraith, Richard C., and Benton J. Underwood. "Perceived Frequency of Concrete and Abstract Words." Memory & Cognition 1, no. 1 (March 1, 1973): 56–60. doi:10.3758/BF03198068. Garnham, Alan, and Jane Oakhill. Thinking and Reasoning. Vol. xiii. Cambridge, MA, US: Basil Blackwell, 1994. Gawronski, Bertram, and Galen V. Bodenhausen. "Associative and Propositional Processes in Evaluation: An Integrative Review of Implicit and Explicit Attitude Change." Psychological Bulletin 132, no. 5 (2006): 692–731. doi:10.1037/0033-2909.132.5.692. Gawronski, Bertram, Fritz Strack, and Galen V. Bodenhausen. "Attitudes and Cognitive Consistency: The Role of Associative and Propositional Processes." Attitudes: Insights from the New Implicit Measures, 2009, 85–117. !40 "General Test Percentage Distribution of Scores Within Intended Broad Graduate Major Field Based on Seniors and Nonenrolled College Graduates." Educational Testing Service, 2013. http://www.ets.org/ s/gre/pdf/gre_guide_table4.pdf. Gianotti, Lorena RR, Christine Mohr, Diego Pizzagalli, Dietrich Lehmann, and Peter Brugger. "Associative Processing and Paranormal Belief." Psychiatry and Clinical Neurosciences 55, no. 6 (2001): 595–603. Gigerenzer, Gerd. Gut Feelings: The Intelligence of the Unconscious. Viking Press, 2007. ---. "The Psychology of Good Judgment Frequency Formats and Simple Algorithms." Medical Decision Making 16, no. 3 (August 1, 1996): 273–80. doi:10.1177/0272989X9601600312. ---. "Why Heuristics Work." Perspectives on Psychological Science 3, no. 1 (2008): 20–29. Gigerenzer, Gerd, and Henry Brighton. "Homo Heuristicus: Why Biased Minds Make Better Inferences." Topics in Cognitive Science 1, no. 1 (2009): 107–43. doi:10.1111/j.1756-8765.2008.01006.x. Gigerenzer, Gerd, and Wolfgang Gaissmaier. "Heuristic Decision Making." Annual Review of Psychology 62 (2011): 451–82. Gigerenzer, Gerd, and Daniel G. Goldstein. "Reasoning the Fast and Frugal Way: Models of Bounded Rationality." Psychological Review 103, no. 4 (1996): 650–69. doi:10.1037/0033-295X.103.4.650. Gigerenzer, Gerd, and Klaus Hug. "Domain-Specific Reasoning: Social Contracts, Cheating, and Perspective Change." Cognition 43, no. 2 (May 1992): 127–71. doi:10.1016/0010-0277(92)90060-U. Goel, Vinod. "Anatomy of Deductive Reasoning." Trends in Cognitive Sciences 11, no. 10 (October 2007): 435–41. doi:10.1016/j.tics. 2007.09.003. Goel, Vinod, Christian Buchel, Chris Frith, and Raymond J. Dolan. "Dissociation of Mechanisms Underlying Syllogistic Reasoning." NeuroImage 12, no. 5 (November 2000): 504–14. doi:10.1006/nimg. 2000.0636. Goel, Vinod, and Raymond J Dolan. "Differential Involvement of Left Prefrontal Cortexin Inductive and Deductive Reasoning." Cognition 93, no. 3 (October 2004): B109–B121. doi:10.1016/j.cognition.2004.03.001. ---. "Explaining Modulation of Reasoning by Belief." Cognition 87, no. 1 (February 2003): B11–B22. doi:10.1016/ S0010-0277(02)00185-3. !41 ---. "Functional Neuroanatomy of Three-Term Relational Reasoning." Neuropsychologia 39, no. 9 (2001): 901–9. doi: 10.1016/S0028-3932(01)00024-0. ---. "Reciprocal Neural Response within Lateral and Ventral Medial Prefrontal Cortex during Hot and Cold Reasoning." NeuroImage 20, no. 4 (December 2003): 2314–21. doi:10.1016/j.neuroimage. 2003.07.027. Goel, Vinod, Brian Gold, Shitij Kapur, and Sylvain Houle. "Neuroanatomical Correlates of Human Reasoning." Journal of Cognitive Neuroscience 10, no. 3 (May 1, 1998): 293–302. doi: 10.1162/089892998562744. Goel, Vinod, and Jordan Grafman. "Role of the Right Prefrontal Cortex in Ill-Structured Planning." Cognitive Neuropsychology 17, no. 5 (2000): 415–36. doi:10.1080/026432900410775. Goel, Vinod, and Oshin Vartanian. "Dissociating the Roles of Right Ventral Lateral and Dorsal Lateral Prefrontal Cortex in Generation and Maintenance of Hypotheses in Set-Shift Problems." Cerebral Cortex 15, no. 8 (August 1, 2005): 1170–77. doi:10.1093/cercor/bhh217. Goethals, George R., and Richard F. Reckman. "The Perception of Consistency in Attitudes." Journal of Experimental Social Psychology 9, no. 6 (November 1973): 491–501. doi: 10.1016/0022-1031(73)90030-9. Goldstein, Daniel G., and Gerd Gigerenzer. "Models of Ecological Rationality: The Recognition Heuristic." Psychological Review 109, no. 1 (2002): 75–90. doi:10.1037/0033-295X.109.1.75. Goodwin, Geoffrey P., and John M. Darley. "The Perceived Objectivity of Ethical Beliefs: Psychological Findings and Implications for Public Policy." Review of Philosophy and Psychology 1, no. 2 (June 1, 2010): 161–88. doi:10.1007/s13164-009-0013-4. Goodwin, Geoffrey P., and Philip Johnson-Laird. "Reasoning About Relations." Psychological Review 112, no. 2 (2005): 468–93. doi: 10.1037/0033-295X.112.2.468. Gosling, Samuel D, Peter J Rentfrow, and William B Swann Jr. "A Very Brief Measure of the Big-Five Personality Domains." Journal of Research in Personality 37, no. 6 (December 2003): 504–28. doi: 10.1016/S0092-6566(03)00046-1. Graham, Jesse, and Jonathan Haidt. "Beyond Beliefs: Religions Bind Individuals Into Moral Communities." Personality and Social Psychology Review 14, no. 1 (February 1, 2010): 140–50. doi: 10.1177/1088868309353415. !42 Graham, Jesse, Jonathan Haidt, and Brian A. Nosek. "Liberals and Conservatives Rely on Different Sets of Moral Foundations." Journal of Personality and Social Psychology 96, no. 5 (2009): 1029–46. doi:10.1037/a0015141. Graham, Jesse, Brian A. Nosek, Jonathan Haidt, Ravi Iyer, Spassena Koleva, and Peter H. Ditto. "Mapping the Moral Domain." Journal of Personality and Social Psychology 101, no. 2 (2011): 366–85. doi: 10.1037/a0021847. Gray, Heather M., Kurt Gray, and Daniel M. Wegner. "Dimensions of Mind Perception." Science 315, no. 5812 (February 2, 2007): 619– 619. doi:10.1126/science.1134475. Gray, Kurt, Adrianna C. Jenkins, Andrea S. Heberlein, and Daniel M. Wegner. "Distortions of Mind Perception in Psychopathology." Proceedings of the National Academy of Sciences 108, no. 2 (January 11, 2011): 477–79. doi:10.1073/pnas.1015493108. Gray, Kurt, Joshua Knobe, Mark Sheskin, Paul Bloom, and Lisa Feldman Barrett. "More than a Body: Mind Perception and the Nature of Objectification." Journal of Personality and Social Psychology 101, no. 6 (2011): 1207–20. doi:10.1037/a0025883. Gray, Kurt, and Daniel M. Wegner. "Feeling Robots and Human Zombies: Mind Perception and the Uncanny Valley." Cognition 125, no. 1 (October 2012): 125–30. doi:10.1016/j.cognition.2012.06.007. Gray, Kurt, Liane Young, and Adam Waytz. "Mind Perception Is the Essence of Morality." Psychological Inquiry 23, no. 2 (2012): 101– 24. doi:10.1080/1047840X.2012.651387. Green, David W., David E. Over, and Robin A. Pyne. "Probability and Choice in the Selection Task." Thinking & Reasoning 3, no. 3 (1997): 209–35. doi:10.1080/135467897394356. Greeno, James G. "Process of Understanding in Problem Solving." Cognitive Theory 2 (1977): 43–83. Griggs, Richard A. "The Effects of Rule Clarification, Decision Justification, and Selection Instruction on Wason's Abstract Selection Task." In Perspectives on Thinking and Reasoning: Essays in Honour of Peter Wason, edited by S. E. Newstead and J. S. B. T. Evans, 17–39. Hillsdale, NJ, England: Lawrence Erlbaum Associates, Inc, 1995. Haidt, Jonathan. "The Emotional Dog and Its Rational Tail: A Social Intuitionist Approach to Moral Judgment." Psychological Review 108, no. 4 (2001): 814–34. doi:10.1037/0033-295X.108.4.814. Haidt, Jonathan, and Jesse Graham. "Planet of the Durkheimians, Where Community, Authority, and Sacredness Are Foundations of Morali- !43 ty." Social and Psychological Bases of Ideology and System Justification, 2009, 371–401. ---. "When Morality Opposes Justice: Conservatives Have Moral Intuitions That Liberals May Not Recognize." Social Justice Research 20, no. 1 (March 1, 2007): 98–116. doi:10.1007/ s11211-007-0034-z. Haidt, Jonathan, Jesse Graham, and Craig Joseph. "Above and Below Left–Right: Ideological Narratives and Moral Foundations." Psychological Inquiry 20, no. 2–3 (2009): 110–19. doi: 10.1080/10478400903028573. Haidt, Jonathan, and Craig Joseph. "Intuitive Ethics: How Innately Prepared Intuitions Generate Culturally Variable Virtues." Daedalus 133, no. 4 (September 1, 2004): 55–66. doi: 10.1162/0011526042365555. Hammond, Kenneth R. "Upon Reflection." Thinking & Reasoning 2, no. 2–3 (1996): 239–48. Hardman, David K., and Stephen J. Payne. "Problem Difficulty and Response Format in Syllogistic Reasoning." The Quarterly Journal of Experimental Psychology Section A 48, no. 4 (1995): 945–75. doi: 10.1080/14640749508401424. Hare, R. M. The Language of Morals. London; New York: Oxford University Press, USA, 1991. Harman, Gilbert. "Change in View: Principles of Reasoning." In Reasoning: Studies of Human Inference and Its Foundations, edited by J. E. Adler and L. J. Rips, 35–46. New York, NY, US: Cambridge University Press, 2008. Hastie, Trevor, Robert Tibshirani, and Jerome Friedman. "The Elements of Statistical Learning. 2001." NY Springer, 2001. Hauser, Marc, Fiery Cushman, Liane Young, R. Kang-Xing Jin, and John Mikhail. "A Dissociation Between Moral Judgments and Justifications." Mind & Language 22, no. 1 (February 1, 2007): 1–21. doi: 10.1111/j.1468-0017.2006.00297.x. Hertwig, Ralph, and Peter M Todd. "More Is Not Always Better: The Benefits of Cognitive Limits." Thinking: Psychological Perspectives on Reasoning, Judgment and Decision Making, 2003, 213–31. Hogarth, Robin M. "Cognitive Processes and the Assessment of Subjective Probability Distributions." Journal of the American Statistical Association 70, no. 350 (June 1, 1975): 271–89. doi: 10.2307/2285808. !44 Holyoak, Keith James, and Robert G Morrison. The Cambridge Handbook of Thinking and Reasoning. New York: Cambridge University Press, 2005. Howe, Mark L, and F. Michael Rabinowitz. "Gist Another Panacea? Or Just the Illusion of Inclusion." Developmental Review 11, no. 4 (December 1991): 305–16. doi:10.1016/0273-2297(91)90016-H. Huemer, Michael. Ethical Intuitionism. Palgrave Macmillan New York, 2005. ---. "Revisionary Intuitionism." Social Philosophy and Policy 25, no. 01 (2008): 368–92. ---. "Transitivity, Comparative Value, and the Methods of Ethics." Ethics 123, no. 2 (January 1, 2013): 318–45. doi:10.1086/668905. Hyde, Janet S, Sara M Lindberg, Marcia C Linn, Amy B Ellis, and Caroline C Williams. "Gender Similarities Characterize Math Performance." Science 321, no. 5888 (2008): 494–95. Hyde, Janet S., Elizabeth Fennema, and Susan J. Lamon. "Gender Differences in Mathematics Performance: A Meta-Analysis." Psychological Bulletin 107, no. 2 (1990): 139–55. doi: 10.1037/0033-2909.107.2.139. Hyde, Janet S., and Janet E. Mertz. "Gender, Culture, and Mathematics Performance." Proceedings of the National Academy of Sciences 106, no. 22 (June 2, 2009): 8801–7. doi:10.1073/pnas.0901265106. Hyde, Janet Shibley. "The Gender Similarities Hypothesis." American Psychologist 60, no. 6 (2005): 581–92. doi:10.1037/0003-066X. 60.6.581. Hyde, Janet Shibley, Elizabeth Fennema, Marilyn Ryan, Laurie A. Frost, and Carolyn Hopp. "Gender Comparisons of Mathematics Attitudes and Affect." Psychology of Women Quarterly 14, no. 3 (September 1, 1990): 299–324. doi:10.1111/j. 1471-6402.1990.tb00022.x. Inbar, Yoel, David Pizarro, and Paul Bloom. "Conservatives Are More Easily Disgusted than Liberals." Cognition & Emotion 23, no. 4 (2009): 714–25. doi:10.1080/02699930802110007. Inbar, Yoel, David Pizarro, Ravi Iyer, and Jonathan Haidt. "Disgust Sensitivity, Political Conservatism, and Voting." Social Psychological and Personality Science 3, no. 5 (September 1, 2012): 537–44. doi: 10.1177/1948550611429024. Ioannidis, John P. A. "Why Most Published Research Findings Are False." PLoS Med 2, no. 8 (August 30, 2005): e124. doi:10.1371/journal.pmed.0020124. !45 Iyer, Ravi, Spassena Koleva, Jesse Graham, Peter Ditto, and Jonathan Haidt. "Understanding Libertarian Morality: The Psychological Dispositions of Self-Identified Libertarians." PLoS ONE 7, no. 8 (August 21, 2012): e42366. doi:10.1371/journal.pone.0042366. Jack, Anthony I. "Introspection: The Tipping Point." Consciousness and Cognition 22, no. 2 (June 2013): 670–71. doi:10.1016/j.concog. 2013.03.005. Jack, Anthony I., Abigail Dawson, Katelyn Begany, Regina Leckie, Kevin Barry, Angela Ciccia, and Abraham Snyder. "fMRI Reveals Reciprocal Inhibition between Social and Physical Cognitive Domains." NeuroImage, 2013. Jack, Anthony I., Abigail J. Dawson, and Megan E. Norr. "Seeing Human: Distinct and Overlapping Neural Signatures Associated with Two Forms of Dehumanization." NeuroImage 79 (October 1, 2013): 313–28. doi:10.1016/j.neuroimage.2013.04.109. Jack, Anthony I., and Philip Robbins. "The Phenomenal Stance Revisited." Review of Philosophy and Psychology 3, no. 3 (September 1, 2012): 383–403. doi:10.1007/s13164-012-0104-5. Jack, Anthony I., Philip Robbins, Jared Friedman, and Chris Meyers. "More than a Feeling: Counterintuitive Effects of Compassion on Moral Judgment." Advances in Experimental Philosophy of Mind, 2014, 125. Jack, Anthony I., and Andreas Roepstorff. "Introspection and Cognitive Brain Mapping: From Stimulus–response to Script–report." Trends in Cognitive Sciences 6, no. 8 (August 1, 2002): 333–39. doi: 10.1016/S1364-6613(02)01941-1. ---. "Why Trust the Subject?" Journal of Consciousness Studies 10, no. 9–10 (2003): v–xx. Johansson, Petter, Lars Hall, Sverker Sikström, and Andreas Olsson. "Failure to Detect Mismatches Between Intention and Outcome in a Simple Decision Task." Science 310, no. 5745 (October 7, 2005): 116–19. doi:10.1126/science.1111709. Johnson-Laird, Philip. "Deductive Reasoning." Annual Review of Psychology 50, no. 1 (1999): 109–35. doi:10.1146/annurev.psych. 50.1.109. ---. "Mental Models and Probabilistic Thinking." Cognition 50, no. 1 (1994): 189–209. ---. Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness. Cambridge, Mass.: Harvard University Press, 1983. !46 Johnson-Laird, Philip, and Bruno G. Bara. "Syllogistic Inference." Cognition 16, no. 1 (February 1984): 1–61. doi: 10.1016/0010-0277(84)90035-0. Johnson-Laird, Philip, and Ruth M.J. Byrne. "Conditionals: A Theory of Meaning, Pragmatics, and Inference." Psychological Review 109, no. 4 (2002): 646–78. doi:10.1037/0033-295X.109.4.646. ---. Deduction. Hove, UK; Hillsdale, USA: L. Erlbaum Associates, 1991. Johnson-Laird, Philip, and Yevgeniya Goldvarg. "How to Make the Impossible Seem Possible," 354–57, 1997. Johnson-Laird, Philip, Paolo Legrenzi, Vittorio Girotto, Maria Sonino Legrenzi, and Jean-Paul Caverni. "Naive Probability: A Mental Model Theory of Extensional Reasoning." Psychological Review 106, no. 1 (1999): 62–88. doi:10.1037/0033-295X.106.1.62. Johnson-Laird, Philip, and Fabien Savary. "Illusory Inferences about Probabilities." Acta Psychologica, Contributions to Decision Making II, 93, no. 1–3 (September 1996): 69–90. doi: 10.1016/0001-6918(96)00022-4. Jones, Matt, and Bradley C. Love. "Pinning down the Theoretical Commitments of Bayesian Cognitive Models." Behavioral and Brain Sciences 34, no. 04 (2011): 215–31. doi:10.1017/ S0140525X11001439. Kahneman, Daniel. "A Perspective on Judgment and Choice: Mapping Bounded Rationality." American Psychologist 58, no. 9 (2003): 697. ---. Thinking, Fast and Slow. 1st ed. New York: Farrar, Straus and Giroux, 2011. Kahneman, Daniel, and Shane Frederick. "A Model of Heuristic Judgment." In The Cambridge Handbook of Thinking and Reasoning, edited by K. J. Holyoak and R. G. Morrison, 267–93. New York, NY, US: Cambridge University Press, 2005. ---. "Representativeness Revisited: Attribute Substitution in Intuitive Judgment." In Heuristics and Biases: The Psychology of Intuitive Judgment, edited by T. Gilovich, D. Griffin, and D. Kahneman, 49–81. New York, NY, US: Cambridge University Press, 2002. Kahneman, Daniel, and Amos Tversky. "On the Psychology of Prediction." Psychological Review 80, no. 4 (1973): 237–51. doi: 10.1037/h0034747. ---. "Prospect Theory: An Analysis of Decision under Risk." Econometrica: Journal of the Econometric Society, 1979, 263–91. !47 Kapogiannis, Dimitrios, Aron K. Barbey, Michael Su, Frank Krueger, and Jordan Grafman. "Neuroanatomical Variability of Religiosity." PloS One 4, no. 9 (2009): e7180. Keil, Frank. "Categorisation, Causation, and the Limits of Understanding." Language and Cognitive Processes 18, no. 5–6 (2003): 663–92. doi:10.1080/01690960344000062. ---. "Folkscience: Coarse Interpretations of a Complex Reality." Trends in Cognitive Sciences 7, no. 8 (August 2003): 368–73. doi: 10.1016/S1364-6613(03)00158-X. Keil, Frank, Leonid Rozenblit, and Candice Mills. "What Lies beneath? Understanding the Limits of Understanding." Thinking and Seeing: Visual Metacognition in Adults and Children, 2004, 227–49. Klein, Gary A. Sources of Power: How People Make Decisions. MIT press, 1998. Knobe, Joshua, Wesley Buckwalter, Shaun Nichols, Philip Robbins, Hagop Sarkissian, and Tamler Sommers. "Experimental Philosophy." Annual Review of Psychology 63, no. 1 (2012): 81–99. doi: 10.1146/annurev-psych-120710-100350. Knobe, Joshua, and Shaun Nichols. Experimental Philosophy: Volume 2. Oxford: Oxford University Press, 2013. Knobe, Joshua, and Richard Samuels. "Thinking like a Scientist: Innateness as a Case Study." Cognition 126, no. 1 (January 2013): 72–86. doi:10.1016/j.cognition.2012.09.003. Koenigs, Michael, Liane Young, Ralph Adolphs, Daniel Tranel, Fiery Cushman, Marc Hauser, and Antonio Damasio. "Damage to the Prefrontal Cortex Increases Utilitarian Moral Judgements." Nature 446, no. 7138 (2007): 908–11. Koleva, Spassena P., Jesse Graham, Ravi Iyer, Peter H. Ditto, and Jonathan Haidt. "Tracing the Threads: How Five Moral Concerns (especially Purity) Help Explain Culture War Attitudes." Journal of Research in Personality 46, no. 2 (April 2012): 184–94. doi: 10.1016/j.jrp.2012.01.006. Kornblith, Hilary. "The Role of Intuition in Philosophical Inquiry: An Account with No Unnatural Ingredients." edited by Michael Depaul and William Ramsey, 129–41. Rethinking Intuition: The Psychology of Intuition and Its Role in Philosophical Inquiry, 1998. Kuntz, Joe R., and J. R. C. Kuntz. "Surveying Philosophers about Philosophical Intuition." Review of Philosophy and Psychology 2, no. 4 (2011): 643–65. !48 Langford, Peter E, and Robert Hunting. "A Representational Communication Approach to the Development of Inductive and Deductive Logic." ADVANCES IN PSYCHOLOGY-AMSTERDAM106 (1994): 191–191. LeBel, Etienne P., and Christopher J. Wilbur. "Big Secrets Do Not Necessarily Cause Hills to Appear Steeper." Psychonomic Bulletin & Review, November 16, 2013, 1–5. doi:10.3758/s13423-013-0549-2. Levinson, Stephen C. "Interactional Biases in Human Thinking." Social Intelligence and Interaction, 1995, 221–60. Levinson, Stephen C. "On the Human 'interactional Engine." In N. J. Enfield & S. C. Levinson (Eds.), Roots of, 2006. Lieberman, David A. "Behaviorism and the Mind: A (limited) Call for a Return to Introspection." American Psychologist 34, no. 4 (1979): 319–33. doi:10.1037/0003-066X.34.4.319. Lieberman, Matthew D. "Intuition: A Social Cognitive Neuroscience Approach." Psychological Bulletin 126, no. 1 (2000): 109. Lin, Shuhong, Boaz Keysar, and Nicholas Epley. "Reflexively Mindblind: Using Theory of Mind to Interpret Behavior Requires Effortful Attention." Journal of Experimental Social Psychology 46, no. 3 (May 2010): 551–56. doi:10.1016/j.jesp.2009.12.019. Lipshitz, Raanan, Gary Klein, Judith Orasanu, and Eduardo Salas. "Taking Stock of Naturalistic Decision Making." Journal of Behavioral Decision Making 14, no. 5 (December 1, 2001): 331–52. doi: 10.1002/bdm.381. Livengood, Jonathan, Justin Sytsma, Adam Feltz, Richard Scheines, and Edouard Machery. "Philosophical Temperament." Philosophical Psychology 23, no. 3 (June 2010): 313–30. doi: 10.1080/09515089.2010.490941. Machery, Edouard, Ron Mallon, Shaun Nichols, and Stephen P Stich. "Semantics, Cross-Cultural Style." Cognition 92, no. 3 (July 2004): B1–B12. doi:10.1016/j.cognition.2003.10.003. Machery, Edouard, and Elizabeth O'Neill. Current Controversies in Experimental Philosophy. 1 edition. New York: Routledge, 2014. Macnamara, John Ed, and Gonzalo E Reyes. The Logical Foundations of Cognition. Oxford University Press, 1994. Maier, Norman. "Reasoning in Humans. The Solution of a Problem and Its Appearance in Consciousness." Journal of Comparative Psychology 12, no. 2 (1931): 181–94. doi:10.1037/h0071361. Malka, Ariel, Yphtach Lelkes, Sanjay Srivastava, Adam B. Cohen, and Dale T. Miller. "The Association of Religiosity and Political Con- !49 servatism: The Role of Political Engagement." Political Psychology 33, no. 2 (2012): 275–99. Manktelow, Ken I, and David E Over. "Deontic Reasoning." Perspectives on Thinking and Reasoning: Essays in Honour of Peter Wason, 1995, 91–114. Manz, Wolfgang. "Experiments on Probabilistic Information Processing." Acta Psychologica 34 (1970): 184–200. doi: 10.1016/0001-6918(70)90016-8. Markovits, Henry, and Pierre Barrouillet. "The Development of Conditional Reasoning: A Mental Model Account." Developmental Review 22, no. 1 (March 2002): 5–36. doi:10.1006/drev.2000.0533. Markovits, Henry, and Robert Vachon. "Conditional Reasoning, Representation, and Level of Abstraction." Developmental Psychology 26, no. 6 (1990): 942–51. doi:10.1037/0012-1649.26.6.942. Martignon, Laura, and Ulrich Hoffrage. "Fast, Frugal, and Fit: Simple Heuristics for Paired Comparison." Theory and Decision 52, no. 1 (February 1, 2002): 29–71. doi:10.1023/A:1015516217425. Marzetti Dall'Aste Brandolini, Silva, and Roberto Scazzieri. Fundamental Uncertainty: Rationality and Plausible Reasoning. Basingstoke, Hampshire; New York: Palgrave Macmillan, 2011. McElroy, Todd, and John J. Seta. "Framing Effects: An Analytic–holistic Perspective." Journal of Experimental Social Psychology 39, no. 6 (2003): 610–17. Messick, Samuel, and Ferdinand J. Fritzky. "Dimensions of Analytic Attitude in Cognition and Personality." Journal of Personality 31, no. 3 (September 1, 1963): 346–70. doi:10.1111/j. 1467-6494.1963.tb01304.x. Mill, John Stuart. On the Definition of Political Economy, and of the Method of Investigation Proper to It: Essays on Some Unsettled Questions of Political Economy (1844); On the Logic of the Moral Sciences: A System of Logic (1856). Routledge/Thoemmes, 1997. Miller, Dale T, and Brian R Taylor. "Counterfactual Thought, Regret, and Superstition: How to Avoid Kicking Yourself." What Might Have Been: The Social Psychology of Counterfactual Thinking, 1995, 305–31. Miller, Earl K., and Jonathan D. Cohen. "An Integrative Theory of Prefrontal Cortex Function." Annual Review of Neuroscience 24, no. 1 (2001): 167–202. doi:10.1146/annurev.neuro.24.1.167. Miller, Tracy. "US Religious Landscape Survey Religious Beliefs and Practices: Diverse and Politically Relevant," 2008. !50 Moll, Jorge, and Ricardo de Oliveira-Souza. "Moral Judgments, Emotions and the Utilitarian Brain." Trends in Cognitive Sciences 11, no. 8 (2007): 319–21. Moll, Jorge, Roland Zahn, Ricardo de Oliveira-Souza, Frank Krueger, and Jordan Grafman. "The Neural Basis of Human Moral Cognition." Nature Reviews Neuroscience 6, no. 10 (2005): 799–809. Mondak, Jeffery J., and Damarys Canache. "Personality and Political Culture in the American States." Political Research Quarterly 67, no. 1 (March 1, 2014): 26–41. doi:10.1177/1065912913495112. Nadelhoffer, Thomas, and Adam Feltz. "The Actor–Observer Bias and Moral Intuitions: Adding Fuel to Sinnott-Armstrong's Fire." Neuroethics 1, no. 2 (July 1, 2008): 133–44. doi:10.1007/ s12152-008-9015-7. Nagel, Jennifer. "Defending the Evidential Value of Epistemic Intuitions: A Reply to Stich." Philosophy and Phenomenological Research, 2013. ---. "Epistemic Intuitions." Philosophy Compass 2, no. 6 (November 1, 2007): 792–819. doi:10.1111/j.1747-9991.2007.00104.x. ---. "Intuitions and Experiments: A Defense of the Case Method in Epistemology." Philosophy and Phenomenological Research 85, no. 3 (2012): 495–527. Nagel, Jennifer, Valerie San Juan, and Raymond A. Mar. "Lay Denial of Knowledge for Justified True Beliefs." Cognition 129, no. 3 (December 2013): 652–61. doi:10.1016/j.cognition.2013.02.008. Nagin, Daniel S., and Greg Pogarsky. "An Experimental Investigation of Deterrence: Cheating, Self-Serving Bias, and Impulsivity*." Criminology 41, no. 1 (February 1, 2003): 167–94. doi:10.1111/j. 1745-9125.2003.tb00985.x. Newell, Allen. "You Can't Play 20 Questions with Nature and Win: Projective Comments on the Papers of This Symposium." Computer Science Department, 1973, Paper 2033. Neys, Wim De, Oshin Vartanian, and Vinod Goel. "Smarter Than We Think When Our Brains Detect That We Are Biased." Psychological Science 19, no. 5 (May 1, 2008): 483–89. doi:10.1111/j. 1467-9280.2008.02113.x. Nichols, Shaun. "After Objectivity: An Empirical Study of Moral Judgment." Philosophical Psychology 17, no. 1 (2004): 3–26. doi: 10.1080/0951508042000202354. !51 ---. "Folk Intuitions on Free Will." Journal of Cognition and Culture 6, no. 1 (March 1, 2006): 57–86. doi: 10.1163/156853706776931385. Nichols, Shaun, and Trisha Folds-Bennett. "Are Children Moral Objectivists? Children's Judgments about Moral and Response-Dependent Properties." Cognition 90, no. 2 (December 2003): B23–B32. doi:10.1016/S0010-0277(03)00160-4. Nieswiadomy, Michael. LSAT Scores of Economics Majors: The 2008-2009 Class Update. SSRN Scholarly Paper. Rochester, NY: Social Science Research Network, June 25, 2009. http://papers.ssrn.com/abstract=1430654. Nisbett, Richard E. Rules for Reasoning. Hillsdale, N.J.: L. Erlbaum Associates, 1993. Nisbett, Richard E, and Stuart Valins. "Perceiving the Causes of One's Own Behavior." Attribution: Perceiving the Causes of Behavior, 1972, 63–78. Nisbett, Richard E., and Nancy Bellows. "Verbal Reports about Causal Influences on Social Judgments: Private Access versus Public Theories." Journal of Personality and Social Psychology 35, no. 9 (1977): 613–24. doi:10.1037/0022-3514.35.9.613. Nisbett, Richard E., Kaiping Peng, Incheol Choi, and Ara Norenzayan. "Culture and Systems of Thought: Holistic versus Analytic Cognition." Psychological Review 108, no. 2 (2001): 291–310. doi: 10.1037/0033-295X.108.2.291. Nisbett, Richard E., and Timothy D. Wilson. "Telling More than We Can Know: Verbal Reports on Mental Processes." Psychological Review; Psychological Review 84, no. 3 (1977): 231. ---. "The Halo Effect: Evidence for Unconscious Alteration of Judgments." Journal of Personality and Social Psychology 35, no. 4 (1977): 250–56. doi:10.1037/0022-3514.35.4.250. Oaksford, Mike, and Nick Chater. "Précis of Bayesian Rationality: The Probabilistic Approach to Human Reasoning." Behavioral and Brain Sciences 32, no. 01 (2009): 69–84. doi:10.1017/ S0140525X09000284. Oechssler, Jörg, Andreas Roider, and Patrick W. Schmitz. "Cognitive Abilities and Behavioral Biases." Journal of Economic Behavior & Organization 72, no. 1 (2009): 147–52. Pacini, Rosemary, and Seymour Epstein. "The Relation of Rational and Experiential Information Processing Styles to Personality, Basic Beliefs, and the Ratio-Bias Phenomenon." Journal of Personality and Social Psychology 76, no. 6 (1999): 972. !52 Pahlke, Erin, Janet Shibley Hyde, and Janet E. Mertz. "The Effects of Single-Sex Compared with Coeducational Schooling on Mathematics and Science Achievement: Data from Korea." Journal of Educational Psychology 105, no. 2 (2013): 444–52. doi:10.1037/ a0031857. Paxton, Joseph M., Leo Ungar, and Joshua D. Greene. "Reflection and Reasoning in Moral Judgment." Cognitive Science 36, no. 1 (2012): 163–77. doi:10.1111/j.1551-6709.2011.01210.x. Paxton, Molly, Carrie Figdor, and Valerie Tiberius. "Quantifying the Gender Gap: An Empirical Study of the Underrepresentation of Women in Philosophy." Hypatia 27, no. 4 (November 1, 2012): 949–57. doi:10.1111/j.1527-2001.2012.01306.x. Pennycook, Gordon. "Evidence That Analytic Cognitive Style Influences Religious Belief: Comment on Razmyar and Reeve (2013)." Intelligence 43 (March 2014): 21–26. doi:10.1016/j.intell.2013.12.005. Pennycook, Gordon, James Allan Cheyne, Nathaniel Barr, Derek J. Koehler, and Jonathan A. Fugelsang. "Cognitive Style and Religiosity: The Role of Conflict Detection." Memory & Cognition 42, no. 1 (January 1, 2014): 1–10. doi:10.3758/s13421-013-0340-7. ---. "The Role of Analytic Thinking in Moral Judgements and Values." Thinking & Reasoning 20, no. 2 (2014): 188–214. doi: 10.1080/13546783.2013.865000. Pennycook, Gordon, James Allan Cheyne, Derek J. Koehler, and Jonathan A. Fugelsang. "Belief Bias during Reasoning among Religious Believers and Skeptics." Psychonomic Bulletin & Review 20, no. 4 (August 1, 2013): 806–11. doi:10.3758/s13423-013-0394-3. Pennycook, Gordon, James Allan Cheyne, Paul Seli, Derek J. Koehler, and Jonathan A. Fugelsang. "Analytic Cognitive Style Predicts Religious and Paranormal Belief." Cognition 123, no. 3 (June 2012): 335–46. doi:10.1016/j.cognition.2012.03.003. Pennycook, Gordon, Jonathan A. Fugelsang, and Derek J. Koehler. "Are We Good at Detecting Conflict during Reasoning?" Cognition 124, no. 1 (July 2012): 101–6. doi:10.1016/j.cognition.2012.04.004. Pennycook, Gordon, and Valerie A. Thompson. "Reasoning with Base Rates Is Routine, Relatively Effortless, and Context Dependent." Psychonomic Bulletin & Review 19, no. 3 (June 1, 2012): 528–34. doi:10.3758/s13423-012-0249-3. Pennycook, Gordon, Dries Trippas, Simon J. Handley, and Valerie A. Thompson. "Base Rates: Both Neglected and Intuitive." Journal of Experimental Psychology: Learning, Memory, and Cognition 40, no. 2 (2014): 544–54. doi:10.1037/a0034887. !53 Petitmengin, Claire, Anne Remillieux, Béatrice Cahour, and Shirley Carter-Thomas. "A Gap in Nisbett and Wilson's Findings? A FirstPerson Access to Our Cognitive Processes." Consciousness and Cognition 22, no. 2 (June 2013): 654–69. doi:10.1016/j.concog. 2013.02.004. Piattelli-Palmarini, Massimo, and Martin Gardner. "Inevitable Illusions: How Mistakes of Reason Rule Our Minds." Nature 374, no. 6517 (1995): 25. Piazza, Jared, and Paulo Sousa. "Religiosity, Political Orientation, and Consequentialist Moral Thinking." Social Psychological and Personality Science, June 13, 2013. doi:10.1177/1948550613492826. Polk, Thad A., and Allen Newell. "Deduction as Verbal Reasoning." Psychological Review 102, no. 3 (1995): 533–66. doi: 10.1037/0033-295X.102.3.533. Pollock, John L. Cognitive Carpentry: A Blueprint for How to Build a Person. A Bradford Book, 2003. Posner, Michael I., and Charles R. Snyder. "Attention and Cognitive Control." In Cognitive Psychology: Key Readings, edited by David Balota, 205–23. Hove, East Sussex, UK: Psychology Press, 2004. Preston, Jesse, and Nicholas Epley. "Explanations Versus Applications The Explanatory Power of Valuable Beliefs." Psychological Science 16, no. 10 (2005): 826–32. ---. "Science and God: An Automatic Opposition between Ultimate Explanations." Journal of Experimental Social Psychology 45, no. 1 (2009): 238–41. Prowse Turner, Jamie A., and Valerie A. Thompson. "The Role of Training, Alternative Models, and Logical Necessity in Determining Confidence in Syllogistic Reasoning." Thinking & Reasoning 15, no. 1 (2009): 69–100. doi:10.1080/13546780802619248. Reber, Arthur S. "An Evolutionary Context for the Cognitive Unconscious." Philosophical Psychology 5, no. 1 (1992): 33–51. doi: 10.1080/09515089208573042. ---. "Implicit Learning and Tacit Knowledge." Journal of Experimental Psychology: General 118, no. 3 (1989): 219–35. doi: 10.1037/0096-3445.118.3.219. ---. "The Cognitive Unconscious: An Evolutionary Perspective." Consciousness and Cognition 1, no. 2 (June 1992): 93–133. doi: 10.1016/1053-8100(92)90051-B. !54 Rensink, Ronald A. "Change Detection." Annual Review of Psychology 53, no. 1 (2002): 245–77. doi:10.1146/annurev.psych. 53.100901.135125. Reyna, Valerie F. "Class Inclusion, the Conjunction Fallacy, and Other Cognitive Illusions." Developmental Review 11, no. 4 (December 1991): 317–36. doi:10.1016/0273-2297(91)90017-I. Reyna, Valerie F., and Charles J. Brainerd. "Fuzzy-Trace Theory: An Interim Synthesis." Learning and Individual Differences, Special Issue: Fuzzy-Trace Theory, 7, no. 1 (1995): 1–75. doi: 10.1016/1041-6080(95)90031-4. Rice, Tom W. "Believe It Or Not: Religious and Other Paranormal Beliefs in the United States." Journal for the Scientific Study of Religion 42, no. 1 (March 1, 2003): 95–106. doi:10.1111/1468-5906.00163. Rich, Marvina C. "Verbal Reports on Mental Processes: Issues of Accuracy and Awareness." Journal for the Theory of Social Behaviour 9, no. 1 (March 1, 1979): 29–37. doi:10.1111/j. 1468-5914.1979.tb00412.x. Rips, Lance J. "Inductive Judgments about Natural Categories." Journal of Verbal Learning and Verbal Behavior 14, no. 6 (December 1975): 665–81. doi:10.1016/S0022-5371(75)80055-7. Robbins, Philip, and Anthony I. Jack. "The Phenomenal Stance." Philosophical Studies 127, no. 1 (2006): 59–85. Roepstorff, Andreas, and Chris Frith. "What's at the Top in the Top-down Control of Action? Script-Sharing and 'top-Top' Control of Action in Cognitive Experiments." Psychological Research 68, no. 2–3 (April 1, 2004): 189–98. doi:10.1007/s00426-003-0155-4. Rosenfeld, Howard M., and Donald M. Baer. "Unnoticed Verbal Conditioning of an Aware Experimenter by a More Aware Subject: The Double-Agent Effect." Psychological Review 76, no. 4 (1969): 425–32. doi:10.1037/h0027451. Samuels, Richard. "The Magical Number Two, plus or Minus: DualProcess Theory as a Theory of Cognitive Kinds." In Two Minds: Dual Processes and beyond, 2009, 129–46. Samuels, Richard, Stephen Stich, and Patrice D. Tremoulet. "Rethinking Rationality: From Bleak Implications to Darwinian Modules." In Cognition, Agency and Rationality, edited by Kepa Korta, Ernest Sosa, and Xabier Arrazola, 21–62. Philosophical Studies Series 79. Springer Netherlands, 1999. http://link.springer.com/chapter/ 10.1007/978-94-017-1070-1_3. Santamaria, Carlos, Juan A. Garcia-Madruga, and Philip N. JohnsonLaird. "Reasoning From Double Conditionals: The Effects of Log- !55 ical Structure and Believability." Thinking & Reasoning 4, no. 2 (1998): 97–122. doi:10.1080/135467898394175. Savage, Leonard J. "Elicitation of Personal Probabilities and Expectations." Journal of the American Statistical Association 66, no. 336 (December 1971): 783–801. doi: 10.1080/01621459.1971.10482346. Savitsky, Kenneth, Boaz Keysar, Nicholas Epley, Travis Carter, and Ashley Swanson. "The Closeness-Communication Bias: Increased Egocentrism among Friends versus Strangers." Journal of Experimental Social Psychology 47, no. 1 (January 2011): 269–73. doi: 10.1016/j.jesp.2010.09.005. Schaeken, Walter. Deductive Reasoning and Strategies. Mahwah, N.J.: L. Erlbaum Associates, 2000. Schmidt, Frank L. "The Relative Efficiency of Regression and Simple Unit Predictor Weights in Applied Differential Psychology." Educational and Psychological Measurement 31, no. 3 (1971): 699– 714. doi:10.1177/001316447103100310. Schooler, Lael J., and Ralph Hertwig. "How Forgetting Aids Heuristic Inference." Psychological Review 112, no. 3 (2005): 610–28. doi: 10.1037/0033-295X.112.3.610. Schulz, Eric, Edward T. Cokely, and Adam Feltz. "Persistent Bias in Expert Judgments about Free Will and Moral Responsibility: A Test of the Expertise Defense." Consciousness and Cognition 20, no. 4 (December 2011): 1722–31. doi:10.1016/j.concog.2011.04.007. Schwitzgebel, Eric, and Fiery Cushman. "Expertise in Moral Reasoning? Order Effects on Moral Judgment in Professional Philosophers and Non-Philosophers." Mind & Language 27, no. 2 (2012): 135–53. Sharples, Robert W. Alexander of Aphrodisias on Fate: Text, Translation, and Commentary. Duckworth, 1983. Shenhav, Amitai, David G. Rand, and Joshua D. Greene. "Divine Intuition: Cognitive Style Influences Belief in God." Journal of Experimental Psychology: General 141, no. 3 (2012): 423. Shiffrin, Richard M., and Walter Schneider. "Controlled and Automatic Human Information Processing: Perceptual Learning, Automatic Attending and a General Theory." Psychological Review 84, no. 2 (1977): 127–90. doi:10.1037/0033-295X.84.2.127. Shynkaruk, Jody M., and Valerie A. Thompson. "Confidence and Accuracy in Deductive Reasoning." Memory & Cognition 34, no. 3 (April 1, 2006): 619–32. doi:10.3758/BF03193584. !56 Simmons, Joseph P., and Leif D. Nelson. "Intuitive Confidence: Choosing between Intuitive and Nonintuitive Alternatives." Journal of Experimental Psychology: General 135, no. 3 (2006): 409–28. doi: 10.1037/0096-3445.135.3.409. Sloman, Steven A. "The Empirical Case for Two Systems of Reasoning." Psychological Bulletin 119, no. 1 (1996): 3–22. doi:10.1037/00332909.119.1.3. Smith, Eliot R., and Jamie DeCoster. "Dual-Process Models in Social and Cognitive Psychology: Conceptual Integration and Links to Underlying Memory Systems." Personality and Social Psychology Review 4, no. 2 (May 1, 2000): 108–31. doi:10.1207/S15327957PSPR0402_01. Smith, Eliot R., and Frederick D. Miller. "Limits on Perception of Cognitive Processes: A Reply to Nisbett and Wilson." Psychological Review 85, no. 4 (1978): 355–62. doi:10.1037/0033-295X.85.4.355. Sosa, Ernest. "Experimental Philosophy and Philosophical Intuition." Philosophical Studies 132, no. 1 (December 14, 2006): 99–107. doi:10.1007/s11098-006-9050-3. ---. "Intuitions: Their Nature and Epistemic Efficacy." Grazer Philosophische Studien 74, no. 1 (June 1, 2007): 51–67. Spielberger, Charles D. "The Role of Awareness in Verbal Conditioning." Journal of Personality 30, no. 3 (September 1, 1962): 73–101. doi: 10.1111/j.1467-6494.1962.tb02309.x. Staël von Holstein, Carl-Axel S. "Two Techniques for Assessment of Subjective Probability Distributions - An Experimental Study." Acta Psychologica 35, no. 6 (December 1971): 478–94. doi: 10.1016/0001-6918(71)90005-9. Stanovich, Keith E. "On the Distinction between Rationality and Intelligence: Implications for Understanding Individual Differences in Reasoning." The Oxford Handbook of Thinking and Reasoning, 2012, 343–65. ---. "Rational and Irrational Thought: The Thinking That IQ Tests Miss." Scientific American Mind 20, no. 6 (2009): 34–39. Stanovich, Keith E., and Richard F. West. "Individual Differences in Reasoning: Implications for the Rationality Debate?" Behavioral and Brain Sciences 23, no. 5 (2000): 645–65. Stevenson, Rosemary J., and David E. Over. "Deduction from Uncertain Premises." The Quarterly Journal of Experimental Psychology Section A 48, no. 3 (1995): 613–43. doi: 10.1080/14640749508401408. !57 Stich, Stephen. "Do Different Groups Have Different Epistemic Intuitions? A Reply to Jennifer Nagel1." Philosophy and Phenomenological Research 87, no. 1 (July 1, 2013): 151–78. doi:10.1111/j. 1933-1592.2012.00590.x. Stier, Haya, and Meir Yaish. "Occupational Segregation and Gender Inequality in Job Quality: A Multi-Level Approach." Work, Employment & Society, February 26, 2014, 0950017013510758. doi: 10.1177/0950017013510758. Storms, Michael D., and Richard E. Nisbett. "Insomnia and the Attribution Process." Journal of Personality and Social Psychology 16, no. 2 (1970): 319–28. doi:10.1037/h0029835. Stotz, K., S. Linquist, P. Griffiths, and E. Machery. "Expert Representations of Innateness," forthcoming. Sutherland, Stuart. Irrationality: The Enemy within. Vol. ix. London, England: Constable and Company, 1992. ---. Irrationality. London: Pinter & Martin Ltd., 2012. Swain, Stacey, Joshua Alexander, and Jonathan M. Weinberg. "The Instability of Philosophical Intuitions: Running Hot and Cold on Truetemp." Philosophy and Phenomenological Research 76, no. 1 (2008): 138–55. Sytsma, Justin, and Edouard Machery. "Two Conceptions of Subjective Experience." Philosophical Studies 151, no. 2 (November 1, 2010): 299–327. doi:10.1007/s11098-009-9439-x. Talbot, Brian. "How to Use Intuitions in Philosophy." University of Southern California, 2009. Thompson, Valerie A. "Conditional Reasoning: The Necessary and Sufficient Conditions." Canadian Journal of Experimental Psychology/ Revue Canadienne de Psychologie Expérimentale 49, no. 1 (1995): 1–60. doi:10.1037/1196-1961.49.1.1. ---. "Dual Process Theories: A Metacognitive Perspective." In Two Minds: Dual Processes and Beyond. Oxford University Press, Oxford, 2009. ---. "Reasoning and Rationality," 67:271–271. Ottowa, Ontario: Canadian Psychological Assoc, 2013. ---. "Towards a Metacognitive Dual Process Theory of Conditional Reasoning." Cognition and Conditionals: Probability and Logic in Human Thinking, 2010, 335–54. ---. "Why It Matters The Implications of Autonomous Processes for Dual Process Theories-Commentary on Evans & Stanovich !58 (2013)." Perspectives on Psychological Science 8, no. 3 (May 1, 2013): 253–56. doi:10.1177/1745691613483476. Thompson, Valerie A., Rakefet Ackerman, Yael Sidi, Linden J. Ball, Gordon Pennycook, and Jamie A. Prowse Turner. "The Role of Answer Fluency and Perceptual Fluency in the Monitoring and Control of Reasoning: Reply to." Cognition 128, no. 2 (August 2013): 256– 58. doi:10.1016/j.cognition.2013.03.003. Thompson, Valerie A., Jonathan Evans, and Jamie I. D. Campbell. "Matching Bias on the Selection Task: It's Fast and Feels Good." Thinking & Reasoning 19, no. 3–4 (2013): 431–52. doi: 10.1080/13546783.2013.820220. Thompson, Valerie A., and Stephen C. Johnson. "Conflict, Metacognition, and Analytic Thinking." Thinking & Reasoning 20, no. 2 (2014): 215–44. doi:10.1080/13546783.2013.869763. Thompson, Valerie A., and Kinga Morsanyi. "Analytic Thinking: Do You Feel like It?" Mind & Society 11, no. 1 (June 1, 2012): 93–105. doi:10.1007/s11299-012-0100-6. Thompson, Valerie A., Jamie A. Prowse Turner, and Gordon Pennycook. "Intuition, Reason, and Metacognition." Cognitive Psychology 63, no. 3 (November 2011): 107–40. doi:10.1016/j.cogpsych. 2011.06.001. Thompson, Valerie A., Jamie A. Prowse Turner, Gordon Pennycook, Linden J. Ball, Hannah Brack, Yael Ophir, and Rakefet Ackerman. "The Role of Answer Fluency and Perceptual Fluency as Metacognitive Cues for Initiating Analytic Thinking." Cognition 128, no. 2 (August 2013): 237–51. doi:10.1016/j.cognition.2012.09.012. Tobacyk, Jerome J. "A Revised Paranormal Belief Scale." Unpublished Manuscript, Louisiana Tech University, Ruston, LA, 1988. http:// thewhitecrow.webs.com/A%20Revised%20Paranormal%20Belief%20Scale.pdf. Tobia, Kevin P., Wesley Buckwalter, and Stephen Stich. "Moral Intuitions: Are Philosophers Experts?" Philosophical Psychology, no. aheadof-print (2012): 1–10. Tobia, Kevin P., Gretchen B. Chapman, and Stephen Stich. "Cleanliness Is Next to Morality, Even for Philosophers," November 26, 2013. http://papers.ssrn.com/abstract=2360029. Todd, Peter M., and Gerd Gigerenzer. "Environments That Make Us Smart Ecological Rationality." Current Directions in Psychological Science 16, no. 3 (June 1, 2007): 167–71. doi:10.1111/j. 1467-8721.2007.00497.x. !59 Tusche, Anita, Thorsten Kahnt, David Wisniewski, and John-Dylan Haynes. "Automatic Processing of Political Preferences in the Human Brain." NeuroImage, 2013. Tversky, Amos. "Assessing Uncertainty." Journal of the Royal Statistical Society. Series B (Methodological) 36, no. 2 (January 1, 1974): 148–59. Tversky, Amos, and Daniel Kahneman. "Availability: A Heuristic for Judging Frequency and Probability." Cognitive Psychology 5, no. 2 (1973): 207–32. ---. "Judgment under Uncertainty: Heuristics and Biases." Science 185, no. 4157 (September 27, 1974): 1124–31. doi:10.1126/science.185.4157.1124. Vaesen, Krist, Martin Peterson, and Bart Van Bezooijen. "The Reliability of Armchair Intuitions." Metaphilosophy 44, no. 5 (October 1, 2013): 559–78. doi:10.1111/meta.12060. Waldmann, Michael R., Jonas Nagel, and Alex Wiegmann. "Moral Judgment." The Oxford Handbook of Thinking and Reasoning, 2012, 364–89. Walker, Janet H. "Real-World Variability, Reasonableness Judgments, and Memory Representations for Concepts." Journal of Verbal Learning and Verbal Behavior 14, no. 3 (June 1975): 241–52. doi: 10.1016/S0022-5371(75)80067-3. Wason, Peter C. "Reasoning." In New Horizons in Psychology, 135–51. Harmondsworth, Middx, UK: Penguin, 1966. ---. "Self-Contradictions." Thinking: Readings in Cognitive Science, 1977, 114–28. "Wason Selection Task." Wikipedia, the Free Encyclopedia, March 30, 2014. http://en.wikipedia.org/w/index.php?title=Wason_selection_task&oldid=572490439. Waytz, Adam, John Cacioppo, and Nicholas Epley. "Who Sees Human? The Stability and Importance of Individual Differences in Anthropomorphism." Perspectives on Psychological Science 5, no. 3 (2010): 219–32. Waytz, Adam, and Nicholas Epley. "Social Connection Enables Dehumanization." Journal of Experimental Social Psychology 48, no. 1 (January 2012): 70–76. doi:10.1016/j.jesp.2011.07.012. Waytz, Adam, Kurt Gray, Nicholas Epley, and Daniel M. Wegner. "Causes and Consequences of Mind Perception." Trends in Cognitive Sciences 14, no. 8 (August 2010): 383–88. doi:10.1016/j.tics. 2010.05.006. !60 Waytz, Adam, Joy Heafner, and Nicholas Epley. "The Mind in the Machine: Anthropomorphism Increases Trust in an Autonomous Vehicle." Journal of Experimental Social Psychology 52 (May 2014): 113–17. doi:10.1016/j.jesp.2014.01.005. Waytz, Adam, Carey K. Morewedge, Nicholas Epley, George Monteleone, Jia-Hong Gao, and John T. Cacioppo. "Making Sense by Making Sentient: Effectance Motivation Increases Anthropomorphism." Journal of Personality and Social Psychology 99, no. 3 (2010): 410–35. doi:10.1037/a0020240. Weinberg, Jonathan M. "How to Challenge Intuitions Empirically Without Risking Skepticism." Midwest Studies In Philosophy 31, no. 1 (September 1, 2007): 318–43. doi:10.1111/j. 1475-4975.2007.00157.x. Weinberg, Jonathan M., Chad Gonnerman, Cameron Buckner, and Joshua Alexander. "Are Philosophers Expert Intuiters?" Philosophical Psychology 23, no. 3 (June 2010): 331–55. doi: 10.1080/09515089.2010.490944. Weinberg, Jonathan M., Shaun Nichols, and Stephen Stich. "Normativity and Epistemic Intuitions." Philosophical Topics 29, no. 1–2 (2001): 429–60. Wetherick, N.E., and K.J. Gilhooly. "Syllogistic Reasoning: Effects of Premise Order." Lines of Thinking 1 (1990): 99–108. White, Peter. "Limitations on Verbal Reports of Internal Events: A Refutation of Nisbett and Wilson and of Bem." Psychological Review 87, no. 1 (1980): 105–12. doi:10.1037/0033-295X.87.1.105. Wiegmann, Alex, Yasmina Okan, and Jonas Nagel. "Order Effects in Moral Judgment." Philosophical Psychology 25, no. 6 (2012): 813–36. doi:10.1080/09515089.2011.631995. Williamson, Timothy. "Philosophical Expertise and the Burden of Proof." Metaphilosophy 42, no. 3 (April 1, 2011): 215–29. doi:10.1111/j. 1467-9973.2011.01685.x. Wilson, Timothy D., Samuel Lindsey, and Tonya Y. Schooler. "A Model of Dual Attitudes." Psychological Review 107, no. 1 (2000): 101–26. doi:10.1037/0033-295X.107.1.101. Wilson, Timothy D., Douglas J. Lisle, Jonathan W. Schooler, Sara D. Hodges, Kristen J. Klaaren, and Suzanne J. LaFleur. "Introspecting about Reasons Can Reduce Post-Choice Satisfaction." Personality and Social Psychology Bulletin 19 (1993): 331–331. Wilson, Timothy D., and Jonathan W. Schooler. "Thinking Too Much: Introspection Can Reduce the Quality of Preferences and Decisions." !61 Journal of Personality and Social Psychology 60, no. 2 (1991): 181–92. doi:10.1037/0022-3514.60.2.181. Winkler, Robert L. "The Quantification of Judgment: Some Methodological Suggestions." Journal of the American Statistical Association 62, no. 320 (1967): 1105–20. doi: 10.1080/01621459.1967.10500920. Yang, Yingrui, and Philip Johnson-Laird. "Illusions in Quantified Reasoning: How to Make the Impossible Seem Possible, and Vice Versa." Memory & Cognition 28, no. 3 (May 1, 2000): 452–65. doi: 10.3758/BF03198560. Yang, Yinguri, Martin D.S. Braine, and David P. O'Brien. "Some Empirical Justification of the Mental-Predicate-Logic Model." In Mental Logic, 333–65. Mahwah, New Jersey: Lawrence Erlbaum Associates, 1998. Yong, Ed. "In the Wake of High Profile Controversies, Pyschologists Are Facing up to Problems with Replication." Nature 483 (2012): 298– 300. Young, Liane, Antoine Bechara, Daniel Tranel, Hanna Damasio, Marc Hauser, and Antonio Damasio. "Damage to Ventromedial Prefrontal Cortex Impairs Judgment of Harmful Intent." Neuron 65, no. 6 (March 25, 2010): 845–51. doi:10.1016/j.neuron. 2010.03.003. Young, Liane, Joan Albert Camprodon, Marc Hauser, Alvaro PascualLeone, and Rebecca Saxe. "Disruption of the Right Temporoparietal Junction with Transcranial Magnetic Stimulation Reduces the Role of Beliefs in Moral Judgments." Proceedings of the National Academy of Sciences, March 29, 2010, 200914826. doi:10.1073/ pnas.0914826107. Young, Liane, Fiery Cushman, Marc Hauser, and Rebecca Saxe. "The Neural Basis of the Interaction between Theory of Mind and Moral Judgment." Proceedings of the National Academy of Sciences 104, no. 20 (2007): 8235–40. Young, Liane, Jonathan Scholz, and Rebecca Saxe. "Neural Evidence for 'Intuitive Prosecution': The Use of Mental State Information for Negative Moral Verdicts." Social Neuroscience 6, no. 3 (2011): 302–15. doi:10.1080/17470919.2010.529712. Zell, Ethan, and Zlatan Krizan. "Do People Have Insight Into Their Abilities? A Metasynthesis." Perspectives on Psychological Science 9, no. 2 (March 1, 2014): 111–25. doi:10.1177/1745691613518075. !