To appear in Journal of Cognitive Science Please do not quote without permission. This is not the final version 1 An affective approach to moral motivation Christine Clavien University of Lausanne christine.clavien@unil.ch Abstract: Over the last few years, there has been a surge of work in a new field called "moral psychology", which uses experimental methods to test the psychological processes underlying human moral activity. In this paper, I shall follow this line of approach with the aim of working out a model of how people form value judgements and how they are motivated to act morally. I call this model an "affective picture": 'picture' because it remains strictly at the descriptive level and 'affective' because it has an important role for affects and emotions. This affective picture is grounded on a number of plausible and empirically supported hypotheses. The main idea is that we should distinguish between various kinds of value judgements by focusing on the sort of state of mind people find themselves in while uttering a judgement. "Reasoned judgements" are products of rational considerations and are based on preliminary acceptance of norms and values. On the contrary, "basic value judgements" are affective, primitive and non-reflective ways of assessing the world. As we shall see, this analysis has some consequences for the traditional internalism-externalism debate in philosophy; it highlights the fact that motivation is primarily linked to "basic value judgements" and that the judgements we openly defend might not have a particular effect on our actions, unless we are inclined to have an emotional attitude that conforms to them. Key words: affect, emotion, emotional reaction, externalism, internalism, moral judgement, motivation. 1. THE MOTIVATION PROBLEM Raymond thinks that we have a moral obligation to help people in distress. On a Sunday morning, he comes across a man lying in the street who has obviously been beaten. Although he 2 would have plenty of time to provide help, he does not feel inclined to help in any way and decides to pass the man by, knowing perfectly well that in doing this, he is failing to follow a moral norm he defends. In other words, he consciously acts in a way contrary to what he rationally thinks should be done.1 One might be puzzled by this example. Although perfectly credible, it seems to contradict the wide spread 'internalist view', according to which accepting the authority of a moral claim implies being motivated to act accordingly.2 Internalism contends that there is an internal or necessary connection between moral judgements and the motivation3 to perform the actions prescribed by these judgements. In this context, motivation is to be understood as an internal force that is directed toward some goal and draws the agent toward this goal; it is the fact of being moved to do something. Internalism is compatible with either the view that moral judgements motivate on their own or the view that they are indirectly but necessarily connected 1 This hypothetical situation is reminiscent of a famous study conducted by Darley and Batson (1973). The two psychologists tested whether seminarians were disposed to provide help when passing by a needy person slumped in an alleyway. They found that, when in a hurry for an appointment, only ten percent of the subjects offered aid. Overall – hurry and not-hurry conditions included – no more than forty percent helped. It must be noted however that this study did not test subjects' psychological states. It is unclear whether the seminarians experienced a motivational conflict, which is not the case in the hypothetical example of Raymond. 2 The kind of internalism I have in mind is close to the form Brink or Dancy are interested in. This internalism states that "moral considerations necessarily motivate" (Brink 1989, p. 42). It is "the conception according to which it is impossible for an agent to make a sincere cognitive moral judgement and not to be motivated accordingly" (Dancy 2000, p.21). Some refinements of the internalist thesis integrate the further condition of practical rationality: the internalist connection is said to hold only in practically rational agents (Smith 1996). However, I do not want to address this particular position here, since it shifts the original question and it is not at all clear what is to be understood by "practically rational agent". Instead, I propose to follow Svavarsdottir's suggestion (1999) to simply add an exception clause to the internalist position that excludes from the discussion cases of agents suffering from motivational disorders – such as depression – that affect them more generally. 3 Motivation can be embedded in various states of minds – such as desires, emotional reactions, or evaluative judgements. Therefore, it might not be easily identified with one particular mental state. 3 to motivation. Raymond's behaviour seems however to confirm the opposing 'externalist' position, according to which, although there is a connection between moral judgement and motivation, it is external and contingent; in other words, moral judgements neither have motivational efficacy in themselves, nor are they necessarily connected to motivation. An internalist opposed to this idea of a source of motivation lying outside moral judgements might wonder whether Raymond's example is realistic, or would be tempted to add some more details to the story, such as the fact that Raymond was in fact inclined to help the injured man, but found himself restrained by other selfish motives. These sorts of undermining or re-describing strategies would, of course, be rejected as unfair and question-begging by an externalist. The internalism-externalism debate has a long history and it is not my purpose to recapitulate well-known arguments. My claim is that it is possible both to take Raymond's case seriously and to resolve the debate simply by distinguishing two sorts of judgements: basic and reasoned. Cases like Raymond's behaviour cease to be puzzling, provided we make this crucial distinction. As I shall try to show, internalism is true for one sort of judgement whereas externalism holds for the second sort. More precisely, I will argue that motivation does not come from conceptual moral considerations – reasoned moral judgements – but from fast and non-reflective appraisals of situations, the ones that are responsible for our basic moral judgements. The defence of this thesis will proceed in two steps. To begin with, I shall present and develop a general picture of the way people evaluate situations (sections 2 to 6). As I will try to show, this picture is supported by data stemming from various sciences such as psychology and neuroscience. It is not my goal to find undeniable proofs in these empirical results, but rather to strengthen the case for my affective approach to morality. In a second step, I will show how this affective picture offers a new framework for rethinking the internalism-externalism debate: combined with a Humean account of motivation (section 7) this picture provides a significant 4 challenge to both internalism and externalism (section 8). In the end, provided the "affective picture" correctly depicts human evaluative activity and the Humean hypothesis is convincing, it will become clear why Raymond's decision is both possible and baffling. 2. THE OUTLINES OF AN AFFECTIVE PICTURE In order to solve Raymond's case, I will propose a picture of the psychological mechanisms underlying moral activity, a description of what might happen in our mind when we judge a situation as to be done or to be avoided and of what motivates us to act. This picture is labelled "affective" because it follows a line of thinking about morality that allows an important role for affect and emotions in moral thinking and activity. Nevertheless, for reasons that will become clear in sections 5 and 6, I depart from the most strongly affective views, such as emotivism, which reduces moral judgements to expressions of emotional reactions (Ayer 1946/1936), expressivism, which takes all moral judgements to be expressions of states of mind about emotions (Gibbard 1990), or Jesse Prinz's recent account of moral judgement as action representation plus emotion (2007). It seems to me that these ways of conceiving of moral judgements do not give full credence to the human capacity to engage in conscious processes of inference and to draw conclusions about what is warranted without the help of emotions. As we shall see, the affective view I favour is a more complex picture. Here is a simplified schema of the affective picture that will be detailed in the following sections. It represents the way we evaluate morally relevant situations. For the sake of simplicity, a broad notion of morality will be used in this paper. Morally relevant situations arise in social contexts where the activity of one human being impinges on the needs and wellbeing of other human beings. Moral judgements are judgements about what one should do, 5 about what is right or wrong in the former kinds of situation.4 Readers can decide for themselves which aspect of the affective picture proposed here they consider moral in a more narrow sense.5 The perception of morally relevant situations such as the one depicted here can either lead to a basic affective appraisal (box 1) or to a reasoned action analysis (box 3) or both. The arrows represent causal influences, either direct (plain arrows) or diffuse (dotted arrows). The grey boxes represent activities of the agent and the white boxes represent shaping forces that influence evaluative activity. 4 By making the condition that moral judgements are about what a human being should do regarding the needs and well-being of other human beings, fashion and etiquette value judgements are ruled out. 5 Although this question will not be discussed here, I fully agree with Bucciarelli et al. (2008) that there exist no clear boundaries between moral and non-moral matters. 1) Cognitive understanding of the situation + Basic affective appraisal (emotional reaction) Reflective process Fast appraisal process 2b) Basic value judgement (verbalised expression of an emotional reaction) 3) Conscious action analysis / reflective and justification process / reflection in terms of values and norms 4) Reasoned value judgement 2a) Non-verbal expression B) Shaping forces (emotional reactions, past experiences, social persuasion, psychological biases, individual goals) A) Shaping forces (inherited tendencies, past experiences and thoughts, habits, individual gaols) 6 Roughly, I will first (sections 3-6) argue that there are two distinct processes in moral activity – or, more broadly, in evaluative activity.6 The former, called the "fast appraisal process", is emotional and automated, whereas the latter, called the "reflective process", is more refined and includes conscious7 processes of inference. Thereafter (section 6), I shall claim that only elements of the first process have the power to motivate – i.e. lead to action. In presenting this double model, I shall argue that there are different kinds of judgements, some of which are more reasoned and lack the power to motivate one to act. 3. EMPIRICAL DATA ON MORAL JUDGEMENT In this section, I will present a collection of data derived from psychology and neuroscience that will help making sense of the above schema and the relationships it depicts between basic emotional states and reasoned thoughts. As we shall see, empirical studies help in understanding the relationship between moral judgements and emotions. They also show that moral judgements are often not the result of conscious processes of inference, in which we consciously apply norms or values to situations in order to infer our judgements; rather, it seems that most moral judgements proceed like intuitive, emotional and highly automated appraisals. To begin, a range of studies breaks the old myth of the rationality of our moral mind and supports the above-mentioned distinction between fast appraisal processes and reflective processes. The most well-known studies are due to Haidt (2001; Haidt et al. 1993), who showed that people usually condemn incestuous practices or other taboo violations even when they can find 6 It should be emphasized that neither of the described appraisal processes have to be specific to moral activity. They generally apply to people's thoughts about what should and should not be done. 7 Note that I do not consider the upper part of the schema to be the domain of the unconscious. Most emotional reactions are consciously felt and subjects are often aware of their eliciting cause (Oatley & Johnson-Laird 1996). The boundaries between consciousness and unconsciousness are difficult to draw and I do not intend to take a position on this matter. 7 no justification apart from their impulsive revulsion to it, which, objectively speaking, is not a good reason for condemning a practice... According to Haidt, these results cast doubt on the rationality of morality as a whole in showing that part of our moral activity is largely unaffected by reflective processes. In other words, a large range of moral judgements seem to be grounded on simple, intuitive – possibly emotional – reactions, instead of moral reasons. Following Haidt's line of thought, I propose to take up a hypothesis, according to which these intuitive reactions are in fact emotional reactions;8 they are basic affective appraisals. As we shall see in the next section, this idea of emotions as the main constituent of moral activity is not new. For now, let us consider some more empirical data and see if they can provide useful information about this intuitive and emotional way of evaluating states of facts. Wheatley and Haidt (2005) have performed an experiment with highly hypnotisable participants. Under hypnosis, they were suggested that they would be disgusted whenever they read an arbitrary word – for example the world 'often'. Participants were then asked to make moral judgements about fairly commonplace stories – some involving moral transgressions and others not – which either contained or not the word linked to their hypnotic disgust. The result is striking: when disgust was prompted by the target word, moral condemnation was vividly 8 Haidt defends a similar view. However, instead of emotional reaction, he would prefer to use the notion of intuition, conceived of as "noninferential quasiperceptual process involving sentiments" (Haidt & Bjorklund 2008: 220). Intuition is a tricky concept because it is so widely used despite lack of consensus over its definition; it usually simply refers to something – an idea, a belief, a judgement, etc. – that forces itself upon us without us knowing exactly how and where we got it from. It is not clear however whether intuition is necessarily related to sentiments. In opposition to Haidt, many would break the link with emotions (Hauser 2006) and some would even describe it as a form of reasoning (i.e. Johnson-Laird 2006). In the affective picture of moral activity presented in this paper, the notion of intuition does not play a significant explanatory role, and when used, it simply refers to something – an idea, a belief, a judgement, etc. – that forces itself upon us without us knowing exactly how and where we got it from. 8 heightened. This study shows that artificially elicited emotions modulate judgements and sometimes even generate moral reactions.9 One obvious way to interpret these data is to say that emotional engagement crucially influences at least some moral judgements. However, it must be stated that they do not really prove that emotional reactions have a causal effect on moral judgements, as opposed to being a side effect or themselves caused by the judgements. The latter view is defended by a range of researchers that have conducted similar studies on the trolley and related dilemmas. Here is a summary of their results and the way they interpret them. Various studies (Mikhail 2002; Hauser 2006; Cushman et al. 2006) were designed to reveal the manner in which people justify their moral judgements. Subjects presented with moral dilemmas like the trolley problem were asked which choices they considered morally acceptable and which reprehensible, then asked to justify their judgements. In their analysis, Mikhail, Hauser and colleagues suggest that some rules deeply rooted in our brains guide our judgements without being available to conscious reasoning because, in the justification task, subjects did not always refer to the rules that have driven them to make their judgements. Among these alleged deep-rooted rules, we find the idea that harm intended as the means to a goal is morally worse than equivalent harm foreseen as the side-effect of a goal; or the idea that physical contact to cause harm to another is morally worse than causing equivalent harm to another without physical contact.10 Mikhail (2002) argues that these results speak against the idea that emotional engagement influences moral judgement. Similarly, Hauser (2008 et al.) considers these data on intuitive 9 Similar results have been obtained by Schnall et al. (2008). Subjects were asked to make moral judgements while sitting in a disgusting room or while exposed to bad smell. The results show that subjects who previously scored highly on a "sensitivity to their own bodily sensations test" made more severe moral judgements when confronted with the disgusting conditions. This indicates that people who habitually listen to their bodies are affected in their moral judgements by extraneous feelings of disgust. 10 Along the same lines, see Baron 1998; Nisbett & Wilson 1977. 9 judgements to support the view that emotions follow from our moral judgements, as opposed to preceding them. According to him, a rule-governed system outputs an evaluation and the evaluation then – sometimes, but not always – prompts an emotional response. This rulegoverned system is a sense of right and wrong that has evolved over millions of years and is shared by people around the world; it precedes both judgements and emotions. However, again, I see nothing in the experimental results that would permit proof of this interpretation. It seems that this debate – which is still on-going (see Hauser et al. 2008) –, is based on a failure to see that the data and part of the interpretations presented by both schools of thought are compatible. If we were to consider emotional reactions as evaluative – a view that will be defended in the next section –, both interpretations would come together in the same analysis. The results of all the experiments are fully compatible with the idea that intuitive judgements proceed from the activation of affect programs, that is, emotional reactions.11 Particular classes of mental states – for example, the apprehension of what seems to us to be intended or nonintended harm – elicit typical emotional reactions that give rise to the expression of specific moral evaluations (for a similar analysis, see Haidt & Joseph 2004). Moreover, in case of conflicting emotional reactions, some might systematically be stronger than others. For example, the fact of causing harm to another via physical contact might elicit a stronger negative emotional reaction than equivalent harm caused indirectly. If one wrongdoing is felt as more important than the other this will in turn influence consciously stated moral judgements. Regularity in the strength of our emotional reactions towards certain classes of situation might give the impression that basic moral judgements are governed by deep-rooted rules. But this interpretation results from a tendency to focus attention on the expression of people's evaluations, that is, on their expressed moral judgements; this is exactly the way in which Mikhail, Hauser and colleagues proceed. However, if we think instead in terms of differentiated 11 Note that I do not attempt to render Greene et al.'s view here, but rather to provide an interpretation of their results that might help in resolving the controversy with Hauser. 10 emotional reactions, we can account for these apparent rules without needing the heavy postulate of a specialised moral sense governed by deep-rooted rules (more on this in Prinz 2008). Such an analysis of moral judgements, which assigns an important role to emotions, is confirmed by studies on normal subjects and patients with ventromedial (VM) prefrontal brain damage, that is, patients with a disrupted emotional system. Ciaramelli et al. (2007) have conducted an experiment on the trolley dilemma with VM patients who had retained their intellectual skills, but displayed abnormal moral conduct and a lack of concern for moral rules in their everyday life. As it turned out, even though the patients were perfectly capable of distinguishing moral from non-moral situations, their judgements of the moral situations were biased in comparison with normal subjects. Presumably because of their impaired emotional system, the patients were much more inclined to consider both scenarios of the trolley dilemma – hitting the switch or pushing a person on the track – as equally acceptable. Similar results have been obtained by Koenigs and colleagues (2007): patients with compromised emotional processing produced an abnormally 'utilitarian' pattern of judgements on moral dilemmas (in the same line, see Mendez et al. 2005). These studies provide good reasons to think that a general emotion deficit has important moral repercussions (Prinz 2008), and more precisely, that emotional reactions are an essential ingredient in at least some moral judgements. 4. FAST APPRAISAL PROCESS In the following sections, I shall present in more detail my affective picture, largely inspired by both the results and analyses provided in the previous section, as well as more classical psychological and philosophical writings on emotions. Let us begin with the first side of the picture: the "fast appraisal process". 11 Morally relevant situations elicit in us "basic affective appraisals" (box 1). One might even consider them a form of primitive judgement (de Sousa 1987; Döring 2007).12 I conceive of those appraisals as a sort of emotional reaction with two component parts intrinsically linked to one another: a fast, non-inferential appraisal13 of a state of affairs and a particular affective state. Due to its "appraisal" side, the emotional reaction has an intentional content – in the philosophical sense that it is directed toward an object.14 More precisely, an emotional reaction seems to be a way of paying selective attention to some features of a situation and perceiving them in a particular manner (see de Sousa 1987). It is some kind of primitive, non-reflective way interpreting a situation and this appraisal expresses itself as a pleasant or unpleasant affective state. Taken that organism usually seek pleasant affective states and avoid the unpleasant ones, we might even understand emotional reactions as merely ways of interpreting a situation as "to be avoided" or "to be pursued". This way of interpreting the world depends on various factors (box A): some inherited tendencies, our past experiences, our acquired habits and long standing goals. 12 In current philosophy of emotions, there is an important debate around the question of the link between the evaluative and the affective aspects of emotional reactions. Some authors have it that the former is causally responsible for the latter (Robinson 2005) whereas other defend the idea that both aspects are insolubly linked with each other (Goldie 2000; Döring 2007). The first view takes the affective part as "bodily feelings" that refer to the Jamesian idea of awareness of internal bodily changes – such as muscular reactions or hormonal changes (James 1884); the supporters of the second view prefer to speak of "emotional feelings", which refer to Peter Goldie's notion of "feeling towards" (Goldie 2000); emotional feelings are intentional – in the sense that they are directed at something in the external world – and insolubly linked with the evaluation contained in the emotion. For the purpose of this article, however, it does not seem crucial to take position in this debate. It is sufficient to say that an emotional reaction is affective and evaluative and that both aspects are closely linked to one other. 13 For an analysis of the non-inferential character of emotional appraisal, see Döring (2007). 14 In so far as these emotional reactions have an intentional content, there is a weak sense in which we can say that they are cognitive. This does not imply, however, that they amount to propositional judgements, that is, kinds of beliefs (more on this in Döring, 2007). 12 To a certain extent, the way we appraise morally relevant facts relies on some fast and frugal mechanisms. When we take the time to reflect on these mechanisms, they often appear to work well, although this is not always the case (Lazarus 1991; Ekman, 1999). Thanks to these mental shortcuts, people have broadly the same emotional reactions when they face similar sorts of states of affairs. For instance, we are generally more intensely affected by harmful acts than harmful omissions15; more generally, we are systematically shocked by actions that cause suffering in others (Nichols 2004) as well as disloyal or opportunist behaviour. It seems that there is a set of inherited mechanisms and capacities designed to generate rapid assessments about what is acceptable or not. Besides this automatic aspect, our capabilities for emotional experience are plastic or developmentally open. This means that our emotional reactions are infused by past experiences and thoughts, and by the context in which they occur. In other words, each emotional reaction is particular in that it is embedded in a subject's life with all her cultural and personal contingencies. For example, take a child that has been reproached several times by his parents for having hidden some personal belongings of his one-year-old sister. Say that these reproaches have made him uncomfortable. As a consequence, he might unconsciously associate being reproached with an unpleasant affective state. And since this state is unpleasant, his brain will "mark" – in the sense of Damasio's somatic marker theory (1994) – the reproaches and the situations likely to elicit them as "to be avoided". In this context, we can speak of learned mechanisms – in contrast with purely innate mechanisms – that trigger emotional reactions, which in turn guide one's future appraisals, choices, and behaviour. In our example, whenever the child conceives the desire to do something that he thinks might cause his parents' reproach, that is, hiding his sister's belongings, his brain will reconstitute the somatic marker of the 15 It is worth noting that this is often the case, even when an action would cause less damage than an omission (see Baron 1998; Sunstein 2005). 13 unpleasant affective state – even if less vivid than in the real circumstances – so that he will avoid this action. Moreover, emotional reactions partly depend on the various motivations we have. For example, findings in social psychology show that people are deeply influenced by the desire to maintain pleasant relationships, and that this motive guides their evaluative attitudes as well as the way in which they process information (Chen et al. 1996). More generally, there are plenty of studies on emotional contagion, the tendency to experience emotions that are similar to and influenced by those of others (Simner 1971; Hatfield et al. 1994). To summarise, our social environment shapes our emotional reactions towards consistency with those of our neighbours. To what extent emotions are plastic is not clear. Case by case experimental studies might be the best way of analyzing emotional plasticity. There is, in fact, a wealth of psychological literature on how they are shaped in a cultural context (Gross 2007, Vandekerckhove et al. 2008). Without going into further details, it is important to recognise that their particular fineness of grain will mainly depend on our experiences, intellectual abilities and on the cultural context in which we have grown up. Before concluding this section two more remarks are needed. Firstly, even if we experience emotional reactions as passive phenomena, – they appear suddenly and effortlessly in consciousness and we are not in control of them – it is important to recognise that they can be evoked or modulated by complex thoughts and beliefs; 16 they are usually elicited by an analysis of the situation – which can be conscious or not –, they often rely on the recognition of fine features of a situation, and they are sometimes even produced in the course of rational 16 Sometimes, even single emotional episodes can be modulated by intellectual considerations. To illustrate this point, consider the example of an episode of anger triggered by free-riding behaviour. This state of anger can last for the duration of the decision-making process, while the subject considers various possible retaliatory actions. In the course of this decision-making process, it is possible that the simple fact of imagining the available outcomes of retaliatory actions has an effect on the strength and quality of the subject's anger; the original emotional reaction might, for example, be modulated into bitterness if the expected outcomes are not satisfying (more on this in Goldie 2000). 14 reflections. Secondly, although emotional reactions are appraisals, it might be disputed whether one should classify them under the category of 'judgements'. Emotional reactions provide input to reasoning and to the making of conscious judgements; they dispose us to some beliefs. However, if one is inclined to consider judgements as verbal expressions, emotional reactions would not count as full judgements. 5. BASIC VALUE JUDGEMENTS An emotional reaction can be expressed; there are two ways to do it. The first way is to express it in a non verbal form (box 2a): for example, making an angry face or shouting "Ah!" This corresponds more or less to an emotivist view of a moral judgement (Ayer 1946/1936). This kind of 'judgement' remains, however, below the threshold of what is under our control; it is an expression that is no more than an automatic and direct extension of an internal emotional reaction. The second way to express a basic value judgement is in verbal form (box 2b); emotional reactions can be conceptualised as "It's horrible!" or "That's good!" This second form of expression is of interest because it is not only an expression of approbation or disapprobation – like the non-verbal form – but a consciously expressed statement in which we attribute a value to something. Hence it seems that we actually cognitively build on our emotional reaction and end up with the production of a slightly more complex appraisal: a "verbalised basic judgement". The idea is that in putting words to an emotional reaction, we not only express it, but produce a conceptually articulated judgement. Hereby, we make a first step in the direction of a reflective process in which, as we shall see, we try to find good reasons for our emotional reactions and judgements. However, even if slightly more refined – because, beyond simply expressing an attitude, we attribute a value to a situation –, these basic judgements are very simple and somehow 15 confused; we more or less state that something is good or bad because we emotionally appraise it that way, without providing much precision. Hence, since these simple basic judgements cannot discriminate finely, they remain somewhere between a bare expression of our disapprobation – or approbation – and the assertion of a reasoned normative statement. An important feature of basic value judgements is the fact that they are intrinsically linked to emotions. With this remark, I would like to emphasise the fact that basic moral judgements do not reduce to bare verbal statements. They are instances of a particular type of state of mind, which consists in consciously coming to be aware of the evaluative aspect of one's emotional reaction and expressing it in a verbalised form. There is also a sense in which one can say that basic value judgements are affectively laden; not only do they carry over the evaluative content of emotional reactions, but also their affective aspect. As we shall see in the next section, reasoned moral judgements (box 4) are instances of another type of state of mind, one which is not emotionally laden. It is worth noting here that moral philosophers are often more interested in reasoned judgements than in basic judgements of the sort just described. On the side of moral psychology (i.e. Hauser or Haidt), no clear distinctions between two sorts of judgements is to be found.17 As we shall see in the following sections, the results of empirical studies might be reconciled with philosophical approaches, precisely with the help of such a distinction. Reasoned judgements are the target of the second side of my affective picture to which we now turn. 17 This lack of distinction leads Haidt to provide a very confusing analysis of dual attitude cases. As he writes: "In cases where the reasoned judgment conflicts with a strong intuitive judgment, a person will have a 'dual attitude' in which the reasoned judgment may be expressed verbally, yet the intuitive judgment continues to exist under the surface, discoverable by implicit measures such as the Implicit Association Test." (2008: 193-94). Here, Haidt implicitly acknowledges an important distinction between the two sorts of judgements, but since his causal analysis of moral activity does not permit him to discriminate between them, intuitive judgements – which, according to Haidt's models are conscious states – suddenly become unconscious. 16 6. THE REFLECTIVE PROCESS We regularly face incoherencies among our own basic judgements or disagreements between our own basic judgements and other's judgements and behaviour. Because our welfare and success depend on our capacity to conduct a coherent life coordinated with those of our neighbours, we feel both the need to convince ourselves of the accuracy of our emotional reactions and the wish that everyone would react the way we do in similar circumstances. These aims incite us to engage in a complex activity of reflection upon our basic judgements – what I will call "reflective process". During the reflective process, we try to consolidate or put some order among our basic judgements. This is done in the context of a reflection that integrates our convictions about fundamental values and norms (box 3). Much of this activity seems, in fact, to be regulated by our basic emotional reactions as well as by our past experiences (box B); we usually tend to value states of affairs that trigger positive emotions in us – and reversely with negative emotions (de Sousa 1987; Hookway 2002). Moreover, due to the fact that the reflective process mostly occurs in a social environment where people think together on absent situations and try to converge on their moral ideas (Gibbard 1990), external influences such as the values and judgements defended by our neighbours or that are prevalent in one's cultural environment come into play (Haidt 2001, p. 818-819; Edwards & Von Hippel 1995; Asch 1955). A range of psychological biases, such as conformist or prestige biases, contribute to reinforce this effect (Davis & Rusbult 2001; Sripada & Stich 2005, 150-155; Fessler & Navarrete 2003). The reflective process just described resembles Haidt's model, and I agree with him, except for the fact that he considers that moral reasoning plays a minor role in moral judgement. According to Haidt, with the exception of some well-trained philosophers, moral reasoning has an impact on our judgements only when it comes through the influence of other people. The major part of the reflective process is "post hoc rationalisation" for our intuitive basic judgements. In other words, people do engage in real or fictional moral debates, but most of the 17 time, these efforts aim at reinforcing or transmitting pre-ordained evaluations (Haidt 2001, Haidt & Bjorklund 2008). This seems too strong a claim. In what follows, I will argue that an accurate account of moral judgement should allow more space for rational tools. There is a complex interplay between affective and reflective activities. Not only is the latter partly regulated by the former, but there is also a loop effect, where reflective activity affects our emotions, and so leads to a change in our fast appraisals. For example, being aware of one's own emotional reaction towards a particular situation enables one to reflect on it. This permits the use of various emotion regulation strategies (Gross 2007, Vandekerckhove et al. 2008). These strategies are particularly useful when people become aware that they have no acceptable reason for holding a particular judgement. For example, if I recognise that I have no good reason to condemn incestuous practices performed by adults of a sound mind who use contraceptive methods, I can train myself not to feel outraged by this kind of situation anymore. More generally, in cases of disagreement, open-minded people usually try to put themselves in the other's shoes or ask themselves whether they have overlooked some relevant aspects of the situation. Such an activity is, in fact, an excellent tool for revising moral evaluations and, thereby, resolving conflicts among people who need to interact with each other over a significant period of time. In an interesting fMRI study, Lamm and colleagues (2007) have investigated the effects of perspective taking and cognitive appraisal on people's empathic reactions. They were able to show that adopting the perspective of the other evokes stronger empathic concern, and knowing that a suffering patient is following a beneficial treatment down-regulates one's affective reaction towards her suffering. Apart from this complex back and forth shaping process between affective and reflective activities, we should not forget the straightforward effect of rational activity in forming moral judgements. It does not seem strikingly difficult and unusual to make value judgements in the course of mere conscious processes of inference – and not necessarily after a preliminary emotional judgement has been made, as implied by Haidt's model (for empirical evidence, see 18 Bucciarelli et al. 2008). Very simple processes of inference can be quickly performed18, such as "I condemn her action because she scorns a moral value I abide by". In this context, I take moral condemnation to be an instance of "reasoned value judgement". Reasoned value judgements are intellectual statements on what is right or wrong, produced while the subject is cool-minded. More precisely, I conceive them to be value utterances arising from a particular type of state of mind; a reasoned activity. These judgements maintain no intrinsic link with emotional engagement. They can occur when people post-rationalise their basic value judgements, but also when they think of abstract hypothetical situations – for example, when one thinks of the effects of global warming for future generations – or situations in which they do not feel particularly engaged – for example, when listening on a cruise boat to the life story of a stranger who complains about the behaviour of her husband. Reasoned judgements do enjoy an indirect relationship with emotions because they rely on preliminary choices of moral norms and values, which themselves seem to be deeply influenced by our emotional reactions. However, the fact that these norms and values are recruited in the course of rational considerations does not necessarily activate the emotional reactions that have led us to accept them. There are good reasons to think that reasoned moral judgements are decoupled from emotions. This idea is supported by various empirical data. In a series of papers for example, Aldina Roskies (2003; 2006) has convincingly shown that ventromedial patients – who present significant deficits in emotional processing – make moral judgements even in the absence of emotional reactions.19 Recall also the above-mentioned studies on ventromedial prefrontal patients (Ciaramelli et al. 2007; Koenigs, et al. 2007). It is worth noting here that 18 When analysing their data on how people evaluate morally relevant situations, experimenters often interpret fast judgements as intuitive. However, one should take seriously the possibility that some of these fast judgements are, in fact, reasoned judgements that have been quickly processed. In other words, time does not seem to be a relevant factor for discriminating between occurrences of basic versus reasoned value judgements. 19 Note that my analysis is consistent with Roskies's model according to which the ventromedial cortex is necessary for the acquisition of moral concepts, but not their retention or employment. 19 these studies do not shed light on the question whether there is a univocal type of judgement that persists through VM trauma. However, with my affective picture in mind, I suspect that these patients produce reasoned moral judgements, while being incapable of basic affective appraisal.20 Some data from brain imaging21 research also confirm this analysis. In two fMRI studies (2004, 2001), Greene and colleagues found systematic variations in the engagement of emotions in moral judgements. When people are confronted with moral dilemmas – such as the trolley dilemma or the crying baby dilemma – several networks in the brain light up, including the brain area related to emotions22 and areas known to be associated with working memory, abstract reasoning, and problem solving23. They also found that "personal dilemmas" – the ones in which subjects have to imagine themselves voluntarily harming somebody – are more emotionally charged than dilemmas that are not "personal". In the latter case, emotion-related areas were not significantly active when subjects made their decisions. According to the authors, there are two appraisal mechanisms that make contributions to our moral judgements and they sometimes play a competitive role – what they call a "dual process model". I would go one step further and propose the hypothesis that there are two appraisal mechanisms that produce two types of judgements: the emotion-based mechanism induces basic value judgements, whereas the rational-based mechanism produces reasoned judgements. Each of these judgements arises from a different type of state of mind. Both sorts of judgements can be 20 Moreover, since there is a constant flow of influence between reflective and emotional mental activities, it is to be expected that VM patients' reasoned judgements will become considerably eroded in the long run. 21 It is worth noting that one should always cast a critical eye over neuroimaging studies. Neuroscience is still an emerging research field and its methodology needs clarification. Various methods are used to interpret brain images and there are vigorous debates about their merits and drawbacks (on this topic, see Henson 2006; Vul et al. 2009). 22 The medial prefrontal cortex, the posterior cingulate/precuneus, and the superior temporal sulcus/temperoparietal junction. 23 Different regions, including the dorsolateral prefrontal cortex and the anterior cingulate cortex. 20 held at the same time and may converge – which is typically the case in post-rationalising situations – or contradict each other – which would be a dual attitude case where two judgements compete. At this point, some critical remarks are needed. Highly artificial dilemmas are often used in experiments because they allow for the systematic manipulation of various parameters and for a statistical analysis of the data. However, the unrealistic content of the stories presented to the participants might prove problematic. Trolley-like dilemmas are so far from real-world situations that one can doubt whether their moral component is taken seriously by the subjects of these experiments. The point here is that subjects do not react to morally relevant situations but simply accomplish an intellectual task that refers to unrealistic hypothetical moral situations. This worry becomes particularly vivid if one tries to present these dilemmas in class. At least in my experience, students first react with a hearty laugh before settling to the task of finding the best answer to the question. This seems to indicate that participants in these experiments merely consider themselves to be involved in an intellectual moral reasoning task – which is a very small part of human moral activity.24 Emotions only come into play when part of the task is to imagine oneself doing something morally repellent – such as pushing a fat man under a runaway trolley. In order to define the real contribution of these studies to a general account of human moral activity, further questions should be answered. To what extent is cognitive activity 'artificially' fostered by these experimental conditions? Is there a correlation between participants' responses and the amount of effort they exert to imagine themselves harming other individuals? Are ordinary people capable of producing cool, non-emotionally charged moral judgements when confronted with real situations or with hypothetical but 24 It is worth noting that Haidt's classic test-stories on incest or other repelling behaviours do not fall prey to this criticism; these stories represent a possible reality and are taken more seriously by the subjects. 21 realistic situations,25 that is, situations that could conceivably befall them in the future? This said, one can at least grant that Greene and colleagues have shown that reasoned judgements do not necessarily recruit emotions. Therefore, these authors are right in expressing doubts about an account of moral judgements always ultimately based on emotional reactions. To sum up, in this section I have argued that, although the moral utterances might be similar, people seem to be in different states of mind when they produce reasoned judgements in contrast to basic emotional judgements. In the latter case, judgements are affectively laden, whereas this characteristic need not apply in the former case. This distinction between basic and reasoned judgements does not refer to their semantic content but to the type of psychological state subjects find themselves in while uttering their judgement. The main lines of the affective picture have now been sketched. They would, of course, need to be spelled out in greater detail. However, there is no need to undertake this task here, since the picture as it has been presented is sufficiently detailed for the purpose of addressing the internalism-externalism debate. In what follows, I will present my view on the motivational aspect of moral judgements: the next section provides evidence for the idea that moral motivation is always mediated by an affective response. This will lead me (section 8) to take position in the old philosophical debate. 7. AFFECTIVE HUMEAN HYPOTHESIS Motivational Humeanism, a longstanding and influential philosophical view, has it that moral motivation must depend upon a pre-existing conative state that is usually conceived of in 25 The distinction between hypothetical and real situations should not be underestimated. Situationist psychologists have shown that even people with genuine moral concerns prove incapable of deploying them under certain situations. Zimbardo's prison experiment, for example, shows how people who condemn torture in the abstract, would make use of it when endowed with the authority to torture (http://www.prisonexp.org/). 22 terms of a desire.26 I think that there are good reasons to accept this line of analysis, but with a focus on affective states – in their simple form or embedded in more complex emotional episodes – instead of desires; desires are responsible for moral motivation precisely because they are intrinsically bound to an affective state. Let us call this implemented view an "affective Humean hypothesis", which is in fact closer to what Hume himself defended when he wrote: "It appears evident that the ultimate ends of human actions can never, in any case, be accounted for by reason, but recommend themselves entirely to the sentiments and affections of mankind" (Hume 2006/1751, Appendix I, V). According to Hume, rational considerations can generate by themselves neither affects nor actions.27 In this section, I shall argue in favour of this view. The affective Humean hypothesis appears to be the best possible explanation for a range of empirical data stemming from psychology. Jennifer Beer and colleagues (2003; 2006) studied patients suffering from damage to brain areas known for being importantly involved in emotional processes (orbitofrontal cortex). With use of various parameters, they reveal an association between deficits in the regulation of social behaviour and dysfunctional selfconscious emotions; patients showed no or inappropriate eliciting of emotions such as embarrassment, shame, guilt or pride. Hence, it seems that the capacity to have self-conscious emotions in appropriate circumstances is involved in the regulation of social behaviour.28 More compelling, various studies on psychopaths or patients with ventromedial prefrontal brain lesions reveal that both sorts of subjects show a dysfunction of their affective system, while there is no evidence in important impairments in their reasoning. On one hand, they are perfectly capable of thinking in terms of norms, on the other hand, they demonstrate little 26 We should make note of the fact that Humeanism does not itself commit one to any particular view in the internalism-externalism debate. Although many philosophers who endorse Humeanism also endorse externalism, it need not be the case. Smith (2006/1994), for instance, is a Humean internalist. 27 At best, rational considerations can monitor an affect by taking into account more relevant information about the situation or by considering the short and long term effects of a situation. 28 Other suggestive results have been found by situationist psychologists. It could be shown that positive feeling has a fostering effect on helping behaviour (Doris 2002). 23 concern for social and moral norms and engage in antisocial behaviour – at a much more dramatic level in the case of the psychopaths (Damasio 1994, Blair et al. 2005, Bechara et al. 1996, Gray et al. 2003). These empirical data show evidence for a strong correlation – or possibly for an essential link – between affective processes and moral action. In this way, they prove to be more in tune with a Humean line of thought than with a rationalist view according to which moral beliefs can themselves give rise to motivation (Nagel 1970; Dancy 2000). Moreover, these data add credence to the affective Humean hypothesis because the simplest possible explanation for the observed strong correlation is to consider affective processes as necessary ingredients for decision-making. This is, of course, no knock-down argument, but here is another one in favour of this view. The philosopher, Mameli, (2005), has analysed Damasio's experiments on patients with brain lesions more closely. He notes that besides having social problems, they also have difficulty making simple everyday practical decisions, even though they are capable of engaging in normal inferential reasoning.29 Now, if rational considerations can not move one to act, what else can do the job? Mameli thinks that it is precisely because the patients show deficits in their affective system that they suffer from impairments to their decision-making. He concludes that our ability to make everyday practical choices, including moral decisions, depends on the proper functioning of the affective system. As he puts it: "In humans, the choice between different actions (...) is always determined by the emotional feelings caused by the thought of possible outcomes of possible actions (...) and not by an unemotional cost-benefit analysis" (2005, 171, my italics). One important correlate of this account is that, although moral statements might result from a conscious process of inferences, the choices of actions are not due to this process. Indeed, a process of inference is no more than a succession of beliefs following some rules, 29 It has been shown that, beside deficits in their affective system, psychopaths face impairments in language (abstract semantic categorization tasks), attention and orientation (Kiehl 2006). However, psychopaths are usually capable of inferential reasoning. This is all we need to give credence to Mameli's analysis. 24 which has nothing to do with the affect. In Mameli's words: "Beliefs about what is morally or socially appropriate in certain circumstances do not exert any motivational force on decisionmaking unless they can trigger emotional feelings that motivate one to choose according to the contents of these beliefs" (2005, 171).30 Let us consider two concurrent models and see why they are less plausible than the affective Humean hypothesis. One might advocate the view that an affective moral reaction is no more than an epiphenomenon of an intrinsically motivating moral cognition or belief. According to this view, we see or understand what the right thing to do should be – or at least we have the impression that we know it; this belief motivates us to act and usually elicits an emotional reaction. The problem with this explanation is that it comes with a high price; it opens up more questions than it resolves. Indeed, taking this argumentative path implies the need to throw new light on phenomena that were easily explained with the affective Humean hypothesis. If moral belief or cognition is sufficient to motivate, why do we so often experience outrage or guilt? If these emotions are nothing but epiphenomena, why are they widely shared in the human species? What role do they play if not a motivating one? Why are some types of emotions obviously responsible for our actions – for instance, fear of snakes – while other types lack this causal efficacy – supposedly epiphenomenal moral emotions? Alternatively, one could argue that moral beliefs are motivating states because they are capable themselves – without any further ingredient – of generating a desire (Dancy 2000) or an emotional reaction that in turn will lead to action. Again, this explanatory strategy opens numerous questions. According to this view, there must be processes by which moral beliefs cause the activation of desires or affective systems that ultimately drive action. But this causal 30 Here Mameli does not state the impossibility of being motivated to perform actions with predicted outcomes that trigger bad affective states. These cases are possible under the condition that other predicted outcomes of the action cause good affective states capable of outweighing the bad ones (Mameli 2005, 172). 25 connection does not hold for any belief – for instance a belief about the validity of an argument.31 So how and why do precisely moral beliefs generate desires or emotional reactions? More crucially, advocates of alternative models to the affective Humean view are faced with the burden of explaining why patients with ventromedial brain lesions are apparently capable of thinking that they should act in one particular way32 without being motivated to do it (Roskies 2003; Damasio et al. 1990). In the end, the principle of parsimony imposes itself and speaks in favour of the affective Humean hypothesis: an affective response is a necessary ingredient for action; more precisely, it is the only "pushing" ingredient that humans dispose of. 8. BACK TO THE INTERNALISM-EXTERNALISM DEBATE The affective Humean hypothesis combined with my affective picture allows one to draw some interesting conclusions in the context of the internalism-externalism debate. As we shall now see, the balance will shift in favour of a hybrid position combining internalism and externalism. Applying the Humean hypothesis to the affective picture reveals two important features of value judgements. Firstly, basic value judgements are intrinsically linked to motivation for they result directly from emotional reactions; precisely because they are affective states, emotional reactions are the key factor in moral action. Secondly, as products of rational inference processes, reasoned judgements are not motivating in themselves for they do not have an affective dimension. Motivation must come from an external source, either from a concomitant 31 It might be argued that a belief about the validity of an argument causes the acceptance of that argument, but this can hardly count as an action mediated by some desire or affective system. 32 Some might object that patients with ventromedial cortex lesions are impaired in reporting their own mental states; either they are wrong about their real judgements or they do not sincerely endorse their moral utterances (Cholbi 2006); but this idea has been convincingly challenged by Roskies (2006). 26 basic judgement – in this case gut and reason would point towards the same direction – or from a desire to follow the reasoned judgement – for example, because we want to avoid punishment or to maintain friendly relationships. This analysis provides a simple account of cases of conflict between moral judgements and actions. It allows for the possibility of judging that something should be done and yet failing to do it. But this can only happen in cases where we coldly produce reasoned judgements that do not sufficiently correspond to our emotional appraisal of the situation. Recall the example of Raymond who denies his help to the beaten man. Raymond might have had no particular emotional reaction when faced with the situation – and therefore could not conceive a basic moral judgement in favour of the needy man –, even though he was conscious of the fact that he was going against one of his reasoned judgements. This situation might be due to the fact that Raymond felt very sleepy on this particular day or that his mind was occupied by the thought of his wife asking him a few hours ago for a divorce. This account provides a significant challenge to the widely supported motivational internalism. If I am right, moral judgements and moral considerations in general do not necessarily motivate; it is possible for an agent to make a sincere reasoned moral judgement – in his armchair, Raymond might have elaborate a bundle of judgements regarding how one should behave towards needy persons – and not be motivated accordingly. Externalism, on the other hand, is challenged at the level of basic judgements since, according to the affective picture combined with the Humean view, this sort of judgement necessarily motivates. At first glance, this hybrid position is rather disturbing because it points to the causal inefficiency of paradigmatic moral judgements – the ones that are carefully elaborated in the course of rational considerations. One might then think that moral actions can only become praiseworthy by accident, and therefore, one might ask whether it is legitimate to consider them as moral. This is a much too pessimistic statement. Firstly, these actions are intrinsically linked to basic moral judgements. Moreover, the links between our automated reactions and our 27 reflective activity are stronger than one might expect. De facto, many of our norms, values and reasoned judgements are closely coordinated with our emotional reactions because simple affective states and emotional reactions do shape our reflective evaluations.33 For example, shaping the cognitive understanding of the assessed situation might reveal relevant characteristics that had been overlooked and hereby modify a previous emotional reaction or trigger a new one. More interestingly, even the emotional mechanisms rooted in our brains are open to the shaping effect of reflective activity – this is the loop effect mentioned in section 6. There might even be room for self-manipulation of emotions. For instance, if we want one of our reasoned judgements to become motivating – or at least correlated with a corresponding motivation – we should try to train a corresponding emotional mechanism. To a certain extent, we probably already manipulate our emotions without consciously seeking this goal. Although this point cannot be pursued here, it is an amazingly complex issue that deserves more attention. Through considering this complex entanglement of relations between affective and reflective moral processes, it becomes clear that there is no reason to depreciate the actions that ensue from the former. One interesting upshot of the proposed distinction between two sorts of value judgements is a reframing of the old debate over moral motivation. In the context of the traditional dispute between internalists and externalists, one notices the lack of a distinction between the semantic level and the psychological level. Accounting for moral motivation implies primarily considering people's states of mind and not merely their expressed statements. The same set of expressed statements can possibly be grounded on different sorts of states of minds. I have argued that moral assertions can arise from two sorts of states of mind – basic or reasoned evaluations. If I am correct, the relevant question is no longer whether moral judgements are 33 This is the reason why motivational internalism is convincing at first glance: reasoned judgements seem to be motivating because they are usually correlated with congruent emotional reactions – but only the latter are the vectors of our moral actions. 28 intrinsically motivating, but rather how these two sorts of underlying mental states can be distinguished precisely. CONCLUSION In this paper, I have presented a story of the way I take people to judge and behave morally. This affective picture is largely inspired by carefully designed empirical studies. Rather than adopting a confrontational philosophical attitude towards empirical studies and looking for reasons to doubt the truth or the significance of their conclusions,34 I began from the presupposition that these studies tell us something interesting about the way we think and act morally. I tried to appropriate them in a sketchy affective picture. The proposed account is not safe from alternative interpretations. However at this stage, it at least has the major benefit of integrating a large range of data stemming from different schools in moral psychology (Haidt, Hauser, Greene). A future task would be, of course, to reinforce this picture with more data and stronger arguments. Meanwhile, I hope to have convincingly highlighted two positive effects of affective processes in the context of moral activity. Firstly, they do not merely distort our judgements but shape them in direct – in the case of basic judgements – and indirect ways – in the case of reasoned judgements. Secondly, they are responsible for moral behaviour; basic value 34 One could object that there is too large a gap between the empirical evidence reviewed and the grand scheme built in this paper. However, I do not think that there are so many alternative interpretations available, precisely because the experiments have been carefully and skilfully thought-out. Another more bothering line of objection would consist in challenging the external validity of the experiments themselves, thereby the whole enterprise of building a general account of moral activity would be compromised. My response to this is that the empirical evidence provided so far help us to make first guesses that are more than plausible suppositions about the way moral activity works. These first guesses await further confirmation – or refutation – from future psychological and neurological studies as well as from information provided by other sciences. 29 judgements are intrinsically motivating because they directly result from emotional reactions and inherit their affective component. Moreover, I hope to have shown that significant conclusions can be drawn from this analysis for the debate over moral motivation. The affective picture explains why our moral judgements do not always prescribe what we ultimately choose to do; reasoned judgements – those free of an affective component – fail to motivate when they are not backed up by basic moral judgements. In the end, it appears that the affective view combines the advantages of both motivational internalism and externalism. In a sense, there is a necessary link between action and moral judgement, nevertheless, cases of discrepancy between one's choices and judgements are possible and easily explained.* * I am grateful to Julien Deonna, Ronald de Sousa, Chloe FitzGerald, Philip Kitcher, Peter Goldie, Hichem Naar, Daniel Schulthess and Fabrice Teroni, for their many useful comments and suggestions. REFERENCES Asch, S. (1955). Opinions and Social Pressure. Scientific American, 193, 31-35. Ayer, A. (1946/1936). Language, Truth, and Logic. New York: Dover Publications. Baron, J. (1998). Judgment misguided: Intuition and error in public decision making. Oxford University Press. Bechara, A. Tranel, D., Damasio, H. Damasio, A. (1996). Failure to Respond Autonomically to Anticipated Future Outcomes Following Damage to Prefrontal Cortex. Cerebral Cortex, 6, 215-225. Beer, J., Heerey, E., Keltner, D., Scabini, D., Knight, R. (2003). The Regulatory Function of Self-Conscious Emotion: Insights From Patients With Orbitofrontal Damage. Journal of Personality and Social Psychology, 85, 594-604. Beer, J., John, O., Scabini, D., Knight, R. (2006). Orbitofrontal Cortex and Social Behavior: Integrating SelfMonitoring and Emotion-Cognition Interactions. Journal of Cognitive Neuroscience, 18, 871-879. Blair, J., Mitchell, D., Blair, K. (2005). The Psychopath: Emotion and the brain. Oxford: Blackwell. Brink, D. (1989). Moral Realism and The Foundations of Ethics, Cambridge. New York: Cambridge University Press. Bucciarelli, M., Khemlani, S., Johnson-Laird P.N. (2008). The Psychology of Moral Reasoning. Judgment and Decision Making, 3, 121-139. Chen, S., Shechter, D., Chaiken, S. (1996). Getting at the truth or getting along: accuracyversus impressionmotivated heuristic and systematic processing. Journal of Personality and Social Psychology, 71, 262-275. 30 Ciaramelli, E., Muccioli, M., Ladavas, E., Di Pellegrino, G. (2007). Selective Deficit in Personal Moral Judgment following Damage to Ventromedial Prefrontal Cortex. SCAN, 2, 84-92. Cholbi, M. (2006). Belief Attribution and the Falsification of Motive Internalism. Philosophical Psychology, 19/5, 607-616. Cushman, F., Young, L., Hauser, M. (2006). The Role of Conscious Reasoning and Intuition on Moral Judgments: Testing Three Principles of Harm. Psychological Science, 17/6, 476-477. Damasio, A. (1994). Descartes'Error: Emotion, Reason, and the Human Brain. New York: Putnam. Damasio, A., Tranel, D., Damasio, H. (1990). Individuals with sociopathic behavior caused by frontal damage fail to respond autonomically to social stimuli. Behavioural Brain Research, 41, 81-94. Dancy, J. (2000). Practical Reality. Oxford, New York: Oxford University Press. Darley, J., Batson, C.D. (1973). "From Jerusalem to Jericho": A Study of Situational and Dispositional Variables in Helping Behavior. Journal of Personality and Social Psychology. 27, 100-108. Davis, J., Rusbult, C. (2001). Attitude Alignment in Close Relationships. Journal of Personality and Social Psychology, 81, 65-84. De Sousa, R. (1987). The Rationality of Emotion. Cambridge (MA): MIT Press. Döring, S. (2007). Seeing what to do: Affective perception and rational motivation. Dialectica, 61/3, 363394. Doris, J. (2002). Lack of Character. Cambridge: Cambridge University Press. Ekman, P. (1999). Basic Emotions. in T. Dalgleish, et al., eds, Handbook of Cognition and Emotion. Sussex: John Wiley & Sons, 45-60. Fessler, D., Navarrete, C. (2003). Meat is Good to Taboo. Journal of Cognition and Culture, 3, 1-40. Gibbard, A. (1990). Wise Choices, Apt Feelings: A Theory of Normative Judgment. Cambridge: Harvard University Press. Goldie, P. (2000). The Emotions: A Philosophical Exploration. Oxford; New York: Oxford University Press. Gray, N., MacCulloch, M., Smith J., Morris M., Snowden R. (2003). Forensic Psychology: Violence Viewed by Psychopathic Murderers. Nature, 423, 497-498. Greene, J. D. (2005), Cognitive Neuroscience and the Structure of the Moral Mind, in: P. Carruthers, et al. (eds.), The Innate Mind: Structure and Contents. Oxford ; New York: Oxford University Press, 338-52. Greene, J., Sommerville, B., Nystrom, L., Darley, J., Cohen, J. (2001). An fMRI Investigation of Emotional Engagement in Moral Judgment. Science, 293, 2105-2108. Greene, J., Haidt, J. (2002). How (and Where) does Moral Judgment Work? Trends in Cognitive Sciences, 6, 517-523. Greene, J., Nystrom, L., Engell, A., Darley, J. & Cohen, J. (2004). The Neural Bases of Cognitive Conflict and Control in Moral Judgment. Neuron, 44, 389-400. Gross, J., ed. (2007). Handbook of Emotion Regulation. New York: Guilford Press. Haidt, J., Koller, S. & Dias, M. (1993). Affect, Culture, and Morality, or Is It Wrong to Eat Your Dog? Journal of Personality and Social Psychology, 65, 613-28. Haidt, J. (2000). The Positive Emotion of Elevation. Prevention & Treatment, 3, 1-4. Haidt, J. (2001). The Emotional Dog and Its Rational Tail: A Social Intuitionist Approach to Moral Judgment. Psychological Review, 108, 814-834. Haidt, J. (2003). The Emotional Dog Does Learn New Tricks: A Reply to Pizarro and Bloom. Psychological Review, 110/1, 197-198. Haidt, J., Joseph, C. (2004). Intuitive ethics: how innately prepared intuitions generate culturally variable virtues. Deadalus, Special Issue on Human Nature, 55-66. Haidt, J. & Bjorklund, F. (2008). Social Intuistionists Answer Six Questions About Moral Psychology. in: W. Sinnott-Armstrong, ed., Moral Psychology. Vol.2: The Cognitive Science of Morality: Intuition and Diversity. Cambridge: MIT Press, 181-217. Hatfield, E., Cacioppo, J. T., Rapson, R. L. (1994). Emotional contagion. New York: Cambridge University Press. Hauser, M. (2006). Moral Minds. New York: Harper Collins. Hauser, M., Young, L., Cushman, F. (2008). Reviving Rawls's Linguistic Analogy: Operative Principles and Causal Structure of Moral Actions. In: W. Sinnott-Armstrong, ed., Moral Psychology. Vol.2: The Cognitive Science of Morality: Intuition and Diversity. Cambridge: MIT Press, 107-43. Behavior, 22, 165-196. Henson, R.. (2006). Forward Inference Using Functional Neuroimaging: Dissociations Versus Associations. Trends in Cognitive Sciences, 10/2, 64-69.Hookway, C. (2002). Emotions and Epistemic Evaluations. in: P. Carruthers, et al., eds., Cognitive Basis of Science. Cambridge: Cambridge University Press, 251-262. 31 Hume, D. (2006/1751). Enquiry into the Principles of Morals. eBooks@Adelaide <http://etext.library.adelaide.edu.au/h/hume/david/h92pm/index.html> James, W. (1884). What Is an Emotion?, Mind, 9, 188-205. Johnson-Laird, P.N. (2006). How we reason. Oxford: Oxford University Press. Kiehl, K. (2006). A cognitive neuroscience perspective on psychopathy: Evidence for paralimbic system dysfunction. Psychiatry Research, 142, 107-128. Koenigs, M., Young, L., Adolphs, R., Tranel, D., Cushman, F., Hauser, M. & Damasio, A. (2007). Damage to the Prefrontal Cortex Increases Utilitarian Moral Judgements. Nature, 446, 908-1011. Lamm, C., Batson, C. D., & Decety, J. (2007). The neural substrate of human empathy: Effects of perspective-taking and cognitive appraisal. Journal of Cognitive Neuroscience, 19/1, 42-58. Lazarus, R. (1991). Emotion and Adaptation. New York: Oxford University Press. Mameli, M. (2005). The Role of Emotions in Ecological and Practical Rationality. in: D. Evans & P. Cruse, eds., (1st ed. 2004), Emotion, Evolution and Rationality. Oxford: Oxford University Press, 159-178. Mendez, M. F., Anderson, E., Shapira, J. S. (2005). An investigation of moral judgement in frontotemporal dementia. Cognitive and Behavioral Neurology, 18/4, 193–197. Mikhail, J. (2002). Aspects of the Theory of Moral Cognition: Investigating Intuitive Knowledge of the Prohibition of Intentional Battery and the Principle of Double Effect. Social Science Research Network (online), 1-129 <http://papers.ssrn.com/sol3/DisplayAbstractSearch.cfm> Moskowitz, G., Skurnik, I, Galinsky, A. (1999). The history of dual process notions, and the future of preconscious control. in: S. Chaiken & Y. Trope, eds., Dual process theories in social psychology. New York: Guilford Press, 12-36. Nagel, T. (1970). The Possibility of Altruism. Oxford: Oxford University Press. Nichols, S. (2004). Sentimental Rules: On the Natural Foundations of Moral Judgment. Oxford: Oxford University Press. Nisbett, R., Wilson, T. (1977). Telling More Than We Can Know: Verbal Reports on Mental Processes. Psychological Review, 84, 231–259. Oatley, K., Johnson-Laird, P.N. (1996). The communicative theory of emotions: empirical tests, mental models, and implications for social interaction. In: L.L. Martin & A. Tesser, eds, Striving and feeling: Interactions among goals, affect, and self-regulation. Mahwah, NJ: Erlbaum, 363-393. Prinz, J. (2007). The Emotional Construction of Morals. Oxford ; New York: Oxford University Press. Prinz, J. (2008). Resisting the Linguistic Analogy: A Commentary on Hauser, Young, and Cushman. In: W. Sinnott-Armstrong, ed., Moral Psychology. Vol.2: The Cognitive Science of Morality: Intuition and Diversity, Cambridge: MIT Press, 157-170. Roskies, A. (2003) Are ethical judgments intrinsically motivational? Lessons from "acquired sociopathy. Philosophical Psychology, 16, 51–66. Roskies, A. (2006). Patients With Ventromedial Frontal Damage Have Moral Beliefs. Philosophical Psychology, 19/5, 617-627. Schnall, S., Haidt, J., Clore, G., Jordan, A. (2008). Disgust as Embodied Moral Judgment. Personality and Social Psychology Bulletin, 34, 1096-109. Simner, M., (1971). Newborn's Response to the Cry of Another Infant. Developmental Psychology, 5, 136150. Smith, M. (2006/1994). The Moral Problem. Oxford: Oxford University Press. Sripada, C., Stich, S. (2005). Evolution, Culture and the Irrationality of the Emotions. in: D. Evans & P. Cruse, eds., (1st ed. 2004), Emotion, Evolution and Rationality. Oxford: Oxford University Press, 133158. Sunstein, C. (2005). Moral Heuristics. Behavioral and Brain Sciences, 28, 531-542. Svavarsdottir, S. (1999). Moral Cognitivism and Motivation. The Philosophical Review, 108/2, 161-219. Thagard, P. (1992). Conceptual Revolutions. Princeton: Princeton University Press. Vandekerckhove, M., von Scheve, C., Ismer, S., Jung, S., Kronast, S., eds. (2008). Regulating Emotions: Culture, Social Necessity, and Biological Inheritance. New Perspectives in Cognitive Psychology. Malden, Oxford: Blackwell. Vul, E., Harris, C., Winkielman, P., Pashler, H. (2009). Puzzlingly High Correlations in Fmri Studies of Emotion, Personality, and Social Cognition. Perspectives on Psychological Science, 4, 274-290. Wheatley, T., Haidt, J. (2005). Hypnotically induced disgust makes moral judgments more severe. Psychological Science, 16, 780-784.