For centuries, philosophers have argued about whether and how moral judgments can be justified (Sinnott-Armstrong, 2007). One recently popular anti-skeptical position in these debates is moral intuitionism, which claims that certain moral judgments are self-evident or justified in themselves or immediately, merely on the basis on understanding them without any need for support from inference or argument from other beliefs (Audi, 2013; Hernandez, 2011; Sinnott-Armstrong, 2001; Stratton-Lake, 2020). The moral judgments that are supposed to be justified without need for inferential support2 are often identified as those that result from a process that is reliable in the sense that it is likely to produce a high proportion of moral judgments that are true (Shafer-Landau, 2005) or at least correct in some broad sense (Blackburn, 1996), even if the person who makes the judgment does not have any reason to believe that the process is reliable. Moral intuitionism then depends on the claim that the processes that lead to moral judgments are reliable in this sense.
Recently, several philosophers (including Sinnott-Armstrong, 2008a, Sinnott-Armstrong, 2008b) have argued that framing effects on moral judgments provide evidence against their reliability and, hence, against the claim that any moral judgments are self-evident or justified in themselves. If a process yields one judgment in some frames but another judgment in other equivalent frames, and if these judgments cannot both be true or correct, then the process must have produced an incorrect judgment in at least one of the frames, even if we do not know which one. Thus, a process that yields enough contrary results in different frames that are morally equivalent cannot be reliable in the specified sense. Critics sometimes debate whether semantically-equivalent frames convey equivalent information (e.g., Aczel, Szollosi, & Bago, 2018; Frisch, 1993; Mandel, 2014; McKenzie & Nelson, 2003; Sher & McKenzie, 2006), but effects of frames that are truly equivalent still do show unreliability. The next step in the argument claims that too much unreliability creates a need for some support by inference or argument. Of course, how much unreliability is too much is a normative issue that can depend on what is at stake. The same degree of unreliability might be acceptable when errors would do little harm, but unacceptable when mistakes could break apart families, friends, and societies, as moral judgments sometimes do. When moral judgments are not reliable enough in this normative sense, we should not trust them without independent confirmation, and then they are not justified in themselves, according to these opponents of moral intuitionism.
This study looks at the empirical evidence concerning valence framing effects on moral judgments in order to test the claims that their formation processes are reliable, as assumed by moral intuitionists and denied by their critics. Psychological research cannot show positively that a moral belief is justified or that the processes that yield moral intuitions are reliable, because those claims are normative or moral instead of scientific. Nonetheless, psychological research can show that such a process is unreliable to the extent that moral judgments are affected by factors that both sides of the debate agree are morally irrelevant.
Moral framing effects provide examples of such morally irrelevant factors. Framing effects on moral judgments occur when an individual's moral judgment is affected (a) by the circumstances of that individual (rather than those of the agent whose act is judged) or (b) by the way in which the options are presented (rather than any feature of the actions that are chosen or judged to be morally right or wrong, which could affect whether those actions are right or wrong). The circumstantial kind of moral framing effect (a) occurs, for example, when bad smells or cleaning products in their environment affect people's moral judgments. In a recent meta-analysis, Landy and Goodwin (2015) found that this kind of effect is not as strong as many moral psychologists claim. In contrast, one presentational kind of moral framing effect (b) occurs when people make different moral judgments about the same scenarios just because the scenarios are presented in a different order. This kind of framing effect has been found in several studies, starting with Petrinovich and O'Neill (1996).
Our meta-analysis will instead focus on a different kind of presentational framing effect: valence framing effects. Valence framing effects occur when participants make different choices or judgments depending on whether the options are described in terms of their positive outcomes (e.g. lives saved) or their negative outcomes (e.g. lives lost). A prominent example that spawned much research on this kind of framing effect is Tversky and Kahneman's famous Disease Problem (TKDP), in which individuals are asked to express a preference between two programs that are described differently in different conditions (see box below; Tversky & Kahneman, 1981).Imagine that the U.S. is preparing for the outbreak of an unusual Asian disease, which is expected to kill 600 people. Two alternative programs to combat the disease have been proposed. Assume that the exact scientific estimate of the consequences of the programs are as follows:
If Program A is adopted, 200 people will be saved.
If Program B is adopted, there is 1/3 probability that 600 people will be saved and 2/3 probability that no people will be saved.
Which of the two programs would you favor?
If Program C is adopted, 400 people will die.
If Program D is adopted, there is 1/3 probability that nobody will die and 2/3 probability that 600 people will die.
Which of the two programs would you favor?
Program A in Condition 1 has exactly the same outcome as Program C in Condition 2, since saving 200 out of 600 means that 400 die. Similarly, Program B in Condition 1 has exactly the same outcome as Program D in Condition 2, since a 1/3 probability of saving all 600 equals a 1/3 probability that none die, and a 2/3 probability that none of the 600 are saved equals a 2/3 probability that all 600 die. The descriptions do not change the outcomes. Consequently, if there are no morally relevant differences between the programs, anyone who favors Program A should also favor Program C and anyone who favors Program B should also favor Program D. Surprisingly, Tversky and Kahneman found that 72% favored Program A but only 22% favored Program C. This finding suggests that many subjects were influenced by the framing or description of the programs in negative or positive terms, so it is a valence framing effect.
These results have been replicated often with the TKDP, but it is still not clear whether similar valence framing effects on moral judgments occur consistently with other moral scenarios. Our meta-analysis tries to answer that question by comparing studies of a wide variety of moral scenarios. A moral scenario pair is suitable to show a valence framing effect only if the options are described or framed positively in one scenario but negatively in the other scenario and yet the options in the different scenarios remain equivalent in all morally relevant respects. The results then show a valence framing effect if participants show a bias in favor of an option under one frame (positive or negative) but not under the other frame.
Many studies have individually investigated the impacts of framing effects on human decision-making; moreover, other meta-analyses have been published seeking to estimate the overall effect size of framing effects across studies. Our present meta-analysis contributes to this field of research in several ways. A previous meta-analysis mainly of valence framing effects on moral judgments (Demaree-Cotton, 2016) concludes that moral intuitions are fairly reliable. However, that meta-analysis failed to include many published studies and also suffered from technical flaws (McDonald, Yin, Weese, & Sinnott-Armstrong, 2019), such as concluding that approximately 80% of people's moral intuitions subject to framing effects don't change by taking the difference between the proportion of moral judgments in distinct frames. By taking the difference between frame groups to show the proportion of people whose moral intuitions are changed by the frame, this interpretation obscures differences among subjects and among types of moral intuitions in susceptibility to framing effects. Further, a difference between framing groups (i.e. 80% difference) is not statistically equivalent to a 80% chance that a randomly selected person's moral judgment is determined by the frame. This inference misconstrues how effect sizes found in between-subjects designs are generalized to new populations. Our meta-analysis involves both between- and within-subjects designs for this purpose. The present meta-analysis is more complete and rigorous, and it provides evidence that our moral intuitions are less reliable than Demaree-Cotton suggests. Other meta-analyses examining framing effects are focused primarily on risky decisions involving neuroeconomic decisions such as mixed gambles, which are not exclusively moral decisions (Steiger & Kühberger, 2018; Kühberger, 1998). Finally, a related meta-analysis of moral judgments focuses on the effect of disgust on moral judgments (Landy & Goodwin, 2015). The present meta-analysis is distinct in its exclusive focus on how framing decisions in terms of positive and negative valence impacts moral decisions in particular.