Papers, Please and the Systemic Approach to Engaging Ethical Expertise in Videogames 1. Introduction Papers, Please, by Lucas Pope (2013), explores the story of a nameless citizen assigned the role of customs inspector at the border of the fictional political regime of Arstotzka. As the inspector the player must process the documents of travellers, deciding who to admit and who to reject or detain. There are rules to be followed and moral choices to be made if and when they are bent or broken (Alexander 2013). In this paper we explore the systems, stories and moral themes of Papers, Please (henceforth PP) to illustrate how this game applies the systemic approach of game design (Juul 2002; Adams & Dormans 2012) to the problem of moral and ethical engagement.1 We also contrast the systemic approach of PP with the more common scripted or branching-narrative approach used in games such as Fallout 3 (Bethesda Game Studios 2008) and Mass Effect (BioWare 2007). To make this comparison we will employ the Four Component model of moral expertise developed by psychologists Narvaez, Rest and colleagues (Rest et al. 1999; Narvaez 2008). This model describes moral behaviour in terms of four component skills – moral motivation, moral sensitivity, moral judgement, and moral action – and provides a structured approach to categorising the different aspects of moral expertise that are engaged by videogames, illustrating the impact that the choice of systemic versus scripted gameplay has on the nature of that engagement. The outline of this paper is as follows. First, we provide an overview of "ethically notable videogames" as a concept and subject of academic inquiry, and from there outline the differences 1 As is common practice in philosophy and other disciplines, we shall use "ethics" and "morality" as interchangeable terms here. 2 between a systemic and scripted approach to game design. Next, we detail the story and systems of PP. We then extract from this analysis four key moral themes – dehumanisation, privacy, fairness and loyalty – that a reflective player (our model player in this study) may perceive in the functioning of the game's systems (Sicart 2010). This demonstrates the moral complexity that can be achieved by the systemic approach. We illustrate these themes with statements from the designer (Pope) and player comments drawn from online reviews and forums. We then proceed to analyse the different sorts of moral engagement that arise in PP by drawing on the Four Component model of ethical expertise. We briefly consider, in terms of this model, some of the ways that scripted and systemic approaches can, in general, differ with regard to each of these categories. Finally, we argue that the systemic approach of PP is particularly effective at engaging a player's expertise in moral sensitivity. 2. Background: Games and Morality Questions of morality are a pervasive topic for media and the arts. Across every medium we see artists engaging audiences with challenging moral questions concerning topics such as war, crime, corruption, fidelity and the abuse of power. Morality is crucial to how human beings understand themselves as societies and individuals, and art – from Greek dramas all the way through to modern Japanese role-playing games – has proven to be a particularly effective vehicle for its interrogation and transmission (Cain 2005; Carr 2006a, 2006b; Vaughn 1990). While games with morally-charged themes such as crime and war are common, relatively few invite us to engage deeply with the morality of the worlds they depict or the behaviours they encourage us to adopt. Those that do, particularly in the AAA commercial space, have often suffered defects that undermine the intellectual and emotive impact of their moral content (Sicart 3 2009, p.199; Heron & Belford 2014, p.42). As Stevenson (2010, p. 37) notes: "while certain techniques are gradually beginning to gain support, it is safe to say that contemporary approaches to incorporating ethical ideas within digital games remain in a nascent phase". The history of "ethically notable videogames" – i.e. videogames that "provide opportunities for encouraging ethical reasoning and reflection" (Zagal 2010, p. 30) – is about as long as the history of gaming itself, with the first notable examples appearing in the early 1980s. One of the earliest and oft-cited of these is Ultima IV: Quest of the Avatar (Origin Systems 1985), the goal of which is to cultivate eight "cardinal virtues" and thereby become worthy of the title of Avatar. Another notable 1985 release, Chris Crawford's Balance of Power (Crawford 1985) is a geopolitical simulator that cleverly satirises the futility and moral emptiness of the nuclear arms race between Cold War superpowers. Following the success of Bioware's Star Wars: Knights of the Old Republic in 2003, it became something of a trend for videogames – role-playing games, in particular – to feature "morality meter" systems, in which the player's moral decisions contribute to an overall "morality score" that reflects their moral character. Popular examples of morality meter games include the inFamous series (Sucker Punch 2009-2014), Fallout 3, the Mass Effect series (BioWare 20072012), and the Fable series (Lionhead Studios 2004-2010). Subject to sustained criticism from academics (Sicart 2009; Melenson 2010), critics (Allen 2014; Floyd 2010), and fans (Aristov 2012), morality meters have fallen out of favour and there is now a "growing corpus" of ethically notable games that treat morality with greater care and nuance (Heron & Belford 2014, p.9). Notable recent examples include Papers, Please, The Witcher 3 (CD Projekt Red 2015), This 4 War of Mine (11 bit Studios 2014), The Walking Dead (Telltale Games 2012), and Spec Ops: The Line (Yager Development 2012). The literature on ethically notable games is active and diverse, attracting the interest of philosophers (Cogburn & Silcox 2008), psychologists (Narvaez et al. 2008), cognitive scientists (Gee, 2003), sociologists (Toma 2015), and games scholars (Jenkins 2007; Frasca 2001). In addition to their critical and aesthetic value, a growing number of scholars argue that ethically notable videogames have educational value as promoters of prosocial values and moral development (Anonymous 2010; Koo & Seider 2010; Schrier 2015; Belman & Flanagan 2010). However, while we agree that ethically notable videogames may have educational value, we are less interested in these aspects here, and more interested in ethical engagement with a game for its own sake, whatever the further learning implications are. This is important since the learning potential of games, in comparison to other forms of active (rather than passive) teaching, has been questioned in the "serious games" literature (Sitzmann 2011; Wouters et. al. 2013). We make no claims here about the relative effectiveness of developing ethical expertise through games in comparison with other teaching methods. Multiple frameworks exist for the classification, criticism, and design of ethically notable games, including Flanagan et al.'s Values at Play methodology (2007), Belman and Flanigan's four principles of empathetically engaging game design (2010), Schrier's EPIC framework (2015), and Sicart's "ethical cognitive friction" approach (2010, 2013). We intend to complement these resources (which we cannot examine here in detail) by introducing the Four Component model as a fruitful source of insight into the different aspects of ethical expertise employed in games. The insights furnished by the Four Component model are largely consistent with these existing 5 frameworks, and could feasibly be used in combination with them. We are, for example, in agreement with Sicart's (2013) insightful definition of ethical gameplay as "the result of the ways in which different choices, dilemmas, situations, and contexts are created by the interrelation between the procedural and the semiotic in the context of play" (p. 60) and endorse his view that "to create ethical gameplay, players need to be empowered as agents" (p. 78). One of the chief virtues of the Four Component model is that it provides a framework for designers to create an empowered moral agent by outlining a comprehensive suite of ethical abilities for games to challenge and engage. 3. The stories and systems of Papers, Please Games engage players' moral expertise through stories and systems, and the friction between them (Sicart 2013). The story elements of a game include its depiction of a world, characters and authored plot elements such as dialogue and cutscenes. The systemic elements are the mechanics of the game: the actions that are provided to the player and the rules which determine how the game responds to these actions (Hunicke, LeBlanc, Zubek 2004; Adams & Dormans 2012). These two components work together: the systems create play and the stories lend an interpretation to that play to give it meaning. While games in general have both systems and stories, different approaches to game design can emphasise one or the other element, especially when it comes to moral themes and skills. This difference in emphasis leads to two different approaches to game design (Juul 2002). Following Stevenson's (2010, p. 39) taxonomy of ethical game types, we understand a systemic approach to game design to be one which relies primarily on the emergent dynamics of the game's systems ("the formal game simulation") to generate an experience in our case moral engagement. We understand a scripted approach to game design 6 to be one which relies primarily on the pre-authored progression of the game's story elements to generate the experience.2 We examine first the story and then the systems of PP, before outlining the complex moral themes these give rise to. In this analysis we employ Sicart's distinction between the reactive player and the reflective player (Sicart 2010, pp. 5-7). The reactive player focuses on strategies to optimise the instrumental behaviour of the system, while the reflective player interacts with the system "as mediated by the semantic layer, taking into account both the need for strategy and what that strategy means in a broader personal and cultural sense" (Sicart 2010, p. 8). In this sense, the reactive player is amoral – they do not engage with the moral significance of the gameplay, it is "just a game". We do not disparage such an approach, but it is not our focus here.3 Because our interest is in ethical engagement, we focus on the morally reflective player, who factors moral considerations into their decision-making, to be our "model player" (in the sense of Eco's "model reader" (Eco 1979)). When we refer to the "player" in what follows, we mean such a model player. Pope acknowledges such players as his target audience, saying "On some level I want players to reach a point of self-realization about how good people can be turned into 2 In addition to these two, Stevenson (2010, p.38) has a third category: "static" ethical games, or games that incorporate ethical themes but grant the player little or no ethical agency – e.g. Shadow of the Colossus (Team Ico, 2005). We consider static ethical games a sub-type of the scripted approach. 3 So-called "speedrunning" – in which gamers strive to finish games as quickly as possible within specific parameters – is reactive play in its purest form. Elaborating on the strategy behind his world record holding Papers, Please run, Joel 'Bisqwit' Yliluoma explains that: "In general, I choose my denial/approval options for special entrants ... to maximize the number of entrants I can process during that day. If I seem too heartless when dealing with some particular case, it's just because they talk too much" (Yliluoma, 2014). 7 uncaring cogs". However, he also welcomes more reactive players who "enjoy the core mechanics of checking documents and searching for discrepancies" (Pope, 2015, personal communication). There is a third type of player which is also outside of the scope of our present analysis: the deliberately immoral player. Such players are aware of the moral content of the game, but enjoy roleplaying as a "heartless bureaucrat", as one such player put it (spupy 2013). While this clearly shows us that there are different ways to approach and play PP, our focus below is on a particular type of model player, the morally reflective player. 3.1 The Story of Papers, Please The story elements of PP are limited but important as they help to contextualise and give moral meaning to the game's systems. In PP the player takes on the role of "the Inspector" in a fictional 1980s political regime called Arstotzka. The story starts as the Inspector's name is drawn in Arstotzka's "labor lottery", resulting in his immediate assignment to the newly reopened Grestin border checkpoint which has been closed for the past six years. The post requires the Inspector to leave his home village with his wife, son, mother-in-law and uncle in order to move into an assigned Class 8 Apartment in Grestin. After commencing his work as a border Inspector the game's story arc is fairly linear, but certain choices can lead to the story ending with the Inspector: 1) retaining his job; 2) in prison; 3) executed by the state; 4) blown up by "terrorists"; 5) escaping on forged passports (with or without his family); or 6) becoming a revolutionary agent. The game tells the player almost nothing about the Inspector or his relationship with his family, beyond their reliance on him for food, shelter, medicine and heat. The Inspector's relatives exist only as role-names on a list: "Wife", "Son", "Uncle", "Mother-in-law" and (later) "Niece" (if the 8 Inspector adopts her). The game focuses on the Inspector's work life and the officials and "travellers" who enter his booth. The latter group includes refugees, diplomats, revolutionaries, reporters, job-seekers, criminals, smugglers, holiday-makers, immigrants and visitors. These travellers come exclusively from nine neighbouring fictional repressive states. Each day of gameplay contains both a number of scripted stories and a number of random cases for the Inspector to process. Some of these characters appear over multiple days, although most appear only once. Some of the random stories are repeated numerous times (such as: "I am coming to live with my wife/husband"), whereas other travellers have little or no story at all – they simply hand over their papers. Beyond these daily interactions, the game's story revolves around the Inspector's relationship with the Arstotzkan regime and the shadowy revolutionary group EZIC. EZIC employs lethal means towards its stated end of overthrowing the corrupt Arstotzkan regime and it demands the Inspector's cooperation. If the Inspector fails to regularly cooperate with EZIC, he is eventually targeted for assassination. The Arstotzkan regime will also execute or imprison the Inspector if he shows disloyalty or incompetence. As well as interacting with officials and travellers in the Inspector's booth, the game's story is moved along by official documents, including changes to the rules for processing papers, and flashes of the front page of the daily newspaper. The latter adds some useful background to the ever changing rules, such as news of a trade war, escaped criminals, or an outbreak of infectious disease. Most of the details, however, of the Inspector's personal life and his wider historical situation are left to the player's imagination. This lack of narrative detail is a deliberate design 9 decision (Pope in Alexander (2013)) taken in order to allow the player to "project their own family" onto the game and to avoid the need for historical accuracy. 3.2 The Systems of Papers, Please The core mechanics of the game revolve around checking the various details on the paperwork of a succession of travellers trying to gain entry to Arstotzka via the Inspector's booth. The player's agency is almost exclusively limited to choosing whether to admit, deny or (later) detain these travellers. Unlike in a story-driven game, the player cannot make dialogue choices or otherwise interact meaningfully with the characters encountered. Figure 1. Checking papers late in the game with many rules and documents. 10 Each morning there is a document describing any new rules instituted that day, such as a new piece of required identification. The rules start out simple and grow in complexity as the game progresses (Figure 1). Some rules are only short-lived, others accrete over time. Some rules apply quite often, others only crop up rarely and are easy to forget. Some discrepancies also allow the player to choose to arrest and detain the traveller rather than simply turn them away. As the game progresses additional mechanisms are added to the player's booth, such as the ability to fingerprint and X-ray travellers. Figure 2. The nightly expenses screen. The player is always under time-pressure. Most days have a fixed real-time limit in which to process as many travellers as possible. The player is rewarded with a salary of $5 per person 11 correctly processed. Making a wrong decision (accepting a traveller who should have been rejected or vice-versa) is penalised. The first two mistakes incur no penalty other than lost earnings. The penalty then increases by $5 per mistake for every two mistakes made thereafter. Each night, after work, the player is faced with their accounts: savings, income and expenses (Figure 2). The primary expenses are rent, food and heating, which start at $20, $20 and $10 respectively, but may increase as the game continues. Any money not spent is saved and can be used at a later day. The player may choose not to spend money on food or heat, in which case their family members become hungry or cold. Prolonged hunger and cold lead to sickness and, eventually, death, although it is safe to purchase food or heat on alternating nights to save money. Not paying rent results in the game immediately ending with the Inspector imprisoned for delinquency. These mechanics mean that players must work quickly in order to process enough travellers to meet their daily expenses. In practice, some days are cut short by terrorist attacks and at other times there are extra expenses, so the player needs to maintain a buffer of savings. Early in the game this is not difficult, but with more complex rules come more pressure on the player to work efficiently. In the later stages of the game the player has opportunities to supplement their income with bribes, kickbacks and bonuses for shooting terrorists and detaining travellers. Processing a traveller means checking various documents (passport, ID, entry permit, etc.) against each other and against the traveller's details (face, sex, height, weight) and story (primarily their reason for entering and length of stay). This process is made more challenging in a variety of ways. Handling the documents is clumsy. The player must drag the documents from the counter onto their desk for inspection. Some documents are many pages long and the player 12 must find the right page. There is not always enough space to view everything simultaneously, so the player must shuffle the documents around, which makes it easier to miss important discrepancies. If a discrepancy is noted, the player may simply reject the traveller's visa or (later) activate the discrepancy checking tool. If the discrepancy is confirmed by the tool (which takes up valuable time), the player has the option to interrogate the traveller who may (or may not) offer an explanation resolving the problem. 4. Moral Themes in Paper, Please A number of important moral themes emerge from the systems and stories of PP. We shall explore four of the most prominent here. 4.1 Dehumanisation We dehumanise others when we regard them as mere means or things (Kant 1999), and not as full moral agents with equal dignity and basic human rights (Fine 2010). PP trains players to dehumanise the travellers they process through repetition and by directing players' attention away from the individuals they are dealing with and towards the documents they are processing. This redirection is achieved, not through story elements, but through the game's mechanics: financial penalties and intense time pressure compel players to process documents quickly and correctly in order to have sufficient earnings to survive. The effect is reinforced by keeping the role-play and the narrative deliberately minimal. Most interpersonal interactions are reduced to a sequence of stock repeated phrases. Each encounter becomes more or less identical, with only the travellers' faces and the documents differing. Even if players want to question a traveller further they cannot do so. While in other games this lack of player-led dialogue might seem 13 frustrating, in PP it reinforces the dehumanising process by creating a greater distance between the player and the travellers. This dehumanising process does not happen all at once. On the first day of the game, when time still feels plentiful, the player may find it interesting to listen to travellers tell their stories. Over time, however, time and financial pressures mount and the stories become more repetitive. After hearing yet another story of a man immigrating to live with his wife the player may simply stop caring. As reviewer Parker (2013) puts it: "Papers, Please quickly convinces the player that the optimum strategy is to immediately reject suspicious documents, ignore pleas for mercy or assistance as every doubt entertained is wasted time and a step closer to losing a loved one or ending the game entirely". This role of the thoughtless and efficient bureaucrat clearly evokes what Hannah Arendt famously calls "the banality of evil" (Arendt 1965). Arendt uses this phrase to describe how a man, such as Nazi bureaucrat Adolf Eichmann, could perpetrate great evil in a "thoughtless" manner (Anonymous 2007, p. 721). By this she means that Eichmann could commit moral atrocities while being motivated, not by hatred, but by the same banal concerns that motivate us all, such as to do our job well, provide for our family, and receive peer recognition. Pope (personal communication, 7 February 2015) explains that he "wanted to show how even a good person, who cares about their family and others ... can be turned into uncaring cogs". PP tries to give players a firsthand experience of how this can happen.4 4 Sicart (2009) argues that games which alienate "the player from reflecting about the ethics of their actions [while] outsourcing moral evaluation to a closed, pre-designed system" (p.193) limit the player's ethical agency in a way that recalls the banality of evil: the player is "deprived of 14 Figure 3. A woman without an entry permit begs for leniency. However, other elements of the game challenge these dehumanising pressures. For example, the game uses interface mechanics - movement, contrasting colours and sound prompts - which are usually used to highlight important information, to distract the player from the documents, which are difficult to read, towards the conversations with travellers. These conversations, while repetitive, often contain morally compelling scenarios and themes (Figure 3). One tells the story of a man coming to Arstotzka for life-saving surgery unavailable in his home country, and who will die if he isn't admitted. Another involves a joyful husband who is happy to finally be free of their ethical capacities in favour of a procedural external system that will evaluate their choices" (p.198). What makes Papers, Please remarkable is that it that does this intentionally and very effectively. 15 tyranny and asks you to please let his wife through even though she lacks valid papers. In another scenario, a woman who thinks she is being tricked into working at a brothel asks the Inspector to detain her pimp. And so on. But the dehumanising pressures of the game's mechanics train the player to ignore these scripted elements. Thus the scripted and systemic elements of the game are in tension (Sicart 2013), with the scripted elements seeking to arouse the player's moral attention while the systemic elements seek to suppress it. The game, however, provides no direct feedback or scripted sequences that acknowledge the moral dimension of the player's decisions. For this reason, players may fail to notice this moral dimension. 4.2 Privacy: Sex, Gender and the misuse of Power Each passport has a sex field which is either M or F (i.e. Male or Female) and which the player must check. However both the game (intentionally)5 and the Inspector (probably unintentionally) quickly confuse issues of sex and gender.6 If travellers' appearance (and gender) does not obviously match their documented sex, if for example a male (sex) looks too feminine (gender), a discrepancy is registered. Accepting such a traveller is a violation (which surprisingly reads "Invalid gender", rather than the technically correct "Invalid sex"). The player is therefore encouraged to look out for such discrepancies. If a traveller is interrogated about such a discrepancy the conversation is typically short: "Are you a woman or a man?", the Inspector bluntly asks, and "The document is correct" is a common response. Sometimes the travellers 5 As Pope (personal communication, 7 February 2015) points out: a "1980's communist state having strict definitions of sex and gender isn't a surprise. It contrasts nicely with how we think about the same thing today." 6 Although the sex/gender distinction is sometimes seen as problematic (Friedman 1991), it is nonetheless useful for this discussion. 16 register disbelief and embarrassment at such a confronting question saying: "Why do you ask?' or "What is this question?" Later in the game after a terrorist attack on Day 6, a new mechanic is added in the form of an Xray scanner. Given that the scanner is specifically introduced in response to a terrorist attack, it seems clear (and the relevant daily briefing reinforces this) that the purpose of the scanner is to identify smugglers bringing in contraband and to prevent terrorist attacks. When a traveller's weight shows a discrepancy from their documented weight, which is an indication that they may be carrying contraband, the scanner becomes available. The scanner is available only when a relevant discrepancy has been noted. When the scanner is used, it provides the player with a black-and-white image of the traveller's front and back in full-body nude which also reveals any hidden items carried on their body. 17 Figure 4. The X-ray scanner is used to investigate a discrepancy in sex/gender. Although the scanner is initially justified as a device to stop terrorists, it is soon put to other uses, raising questions about the creeping expansion of security powers. For example, one traveller is an Arstotzkan female (according to the "sex" on her passport) with masculine facial features. If the discrepancy between her image and her passport is interrogated, the option to use the scanner appears. Under interrogation the traveller affirms that her passport is correct, but the scanner reveals that she seems to have male genitals (Figure 4). This is treated as a discrepancy by the game and the player will be penalised if they do not refuse her entry. Whenever the facial features of travellers don't conform to gender stereotypes about what a man or a woman should 18 look like, their sex can be quizzed and their privacy infringed by the use of the scanner to reveal an image of their genitals. Early on it is easy to dismiss such cases as careless forgeries. But after a while the player may begin to suspect that some of these discrepancies are not forgeries, but the result of oppressive and limiting bureaucratic options and the state's confusion between sex and gender. This raises an important moral issue: should players be ordering invasive scans on the basis of a hunch about a traveller's sex based on stereotypical views about masculine or feminine facial features? As one gamer puts it: "What if the ... wrong gender on the person's passport [is] just the rigid rules of society forcing his or her true gender to be displayed on the passport, even though he or she ached to be seen as otherwise?" (Davis, 2013) The invasive nature of the scanner heightens this moral issue. The pictures it produces are graphic full-frontal images and clearly an important invasion of the traveller's privacy. Should the player risk suffering a financial penalty or should they infringe the privacy of vulnerable travellers? The game, however, never explicitly identifies the choice to use the scanner in such cases as a moral issue at all. In this way, the inclusion of the scanner highlights the moral tension between public security and personal privacy. As Pope says, "the scanner's role flops between critically useful and unnecessarily intrusive. It nicely highlights the challenge of keeping people both alive and happy in this kind of situation" (personal communication, 7 February 2015). 4.3 Fairness: Bending the rules to help others Even when the player recognises that a traveller might have a compelling moral reason to be let through with invalid papers (or denied entry with valid papers), the moral issue of whether to 19 bend the rules for them is a complex one. This complexity arises because of the problems of procedural fairness and moral consistency. These problems primarily arise, not from the scripted elements of the game (although these do scaffold and contextualise the issues), but from the game's systems. The player cannot let everyone through, or else the Inspector would lose his job and be imprisoned. This leaves open only two viable strategies. First, to try to always follow the bureaucratic rules as best you can no matter what the content of those rules are and no matter who they are applied to. Second, to mostly apply those rules correctly, but to make a few exceptions on moral (or other, perhaps selfish) grounds. If players adopt the first strategy, then they have chosen to roleplay as a thoughtless or selfish bureaucrat who blindly carries out even immoral and unfair rules. If players adopt the second strategy, the moral problem becomes how to do so fairly and consistently. Players can't always bend the rules fairly since they lack both the information and the time to carefully consider all the relevant facts when making that decision. While some travellers freely offer a (comparatively) detailed and emotional story, many do not, even if their plight is no less worthy of consideration. This is complicated by the player's inability to further interrogate the traveller's story, which makes it impossible to determine who is lying or embellishing. Given this it seems impossible to decide in a fair way who the player should let through on moral grounds. Most likely players will arbitrarily let through those who are able to tell them the most emotionally convincing tale, even though others may be more deserving. In regards to consistency, it will be difficult, unless one's rules or principles are very (and perhaps unfairly) restrictive, to bend the rules in any consistent way. For example, suppose that players decide that they will let through, on moral grounds, travellers who are reuniting with their spouses. But unless this case only occurs once or twice a day, on at least some days, players may not be able 20 to act consistently by letting through every case that fits this rule and still earn a living wage. In a complete game of PP the player processes up to three or four hundred travellers; there is simply no fair and consistent way to deal with all of them. Furthermore, lost income from intentionally admitting people with invalid documents for moral reasons deprives the Inspector and his family of much needed income to provide food and heat. How much of a cost can players reasonably expect the Inspector's family to bear in order to maintain the Inspector's moral integrity? 4.4 Loyalty: who should you support? Both the government and EZIC demand obedience and loyalty from the Inspector. This loyalty, however, is not demanded in a single big decision. Rather the player is gradually drawn into it by a number of smaller choices. While these choices start out easy enough, the demands made on the Inspector quickly increase, making it hard to keep showing loyalty as well as risky to stop doing so. Unlike the first three moral themes, this moral issue arises, not so much out of the game's systems themselves, but out of the interaction between the scripted elements and the game's mechanics. Who (if anyone) should the player be loyal to? With the limited information the player has, it seems that, morally, neither side deserves unconditional loyalty. The Arstotzkan regime is running a repressive and unjust police state, but not all of its rules are unreasonable. For example, when there is a breakout of disease it reasonably enough introduces a rule that travellers must have appropriate vaccinations. However, the regime is also corrupt, and it will imprison the Inspector for trivial reasons (such as missing one day's rent) and execute him if he is disloyal. EZIC claims to fight only to overthrow the corrupt Arstotzkan government, but its true intentions are unknown. Further, while EZIC's demands start out as fairly mild, such as 21 letting an agent through with invalid papers, the player is soon offered large bribes to perform a number of increasingly questionable actions, including murder (involving collateral damage) and self-sacrifice. EZIC will also try to kill the Inspector if he is not loyal to them. Both sides show themselves to be at the very least morally questionable, and yet the player cannot help but take sides (at least implicitly) since either they comply with EZIC's demands or, in effect, they comply with the government's demands (even if they do so for self-interested reasons rather than out of loyalty). Either way players become morally corrupted by what they choose to do. 5. Systemic vs. Scripted approaches to Moral Engagement This analysis of the relatively sparse systems of PP illustrates that even simple systems can generate complex moral themes. But to get an even deeper understanding of the systemic approach of PP we need to do two further things. First, draw some general contrasts between the systemic approach of PP and the more common scripted approach (exhibited in such games as The Walking Dead, Mass Effect, Fallout 3 and many others). Second, to outline the way that these different approaches to game design can lead to different types and degrees of moral engagement. The principal design pattern employed in scripted approaches is the branching narrative (Sweetser & Wiles 2005; Adams 2010). Moral choices are typically presented in the context of a conversation system, characters offer the player different points of view around a moral problem, and the player chooses how to respond. Often the response comes down to a single pivotal decision in which the player chooses who to side with from a list of dialogue choices, or in some cases, a quick time event. The game mechanic behind these choices is simple and generic. Stripped of its story, it boils down to a multiple choice problem with a handful of possible 22 options (Short 2015). The moral weight comes primarily not from the system, but from the scripted elements that come before and after the choice. The game may also have other mechanics for movement, combat, etc., but these rarely interface deeply with the moral choices, except to transport the player from one to the next. In contrast, the systemic approach of PP (which is shared by other games such as September 12th and The McDonald's Game) "handles ethical decision making as an emergent property of the game's ludic elements" (Heron & Belford 2014, p.4) The meaning of the game is less concentrated in the progression of story elements (although they still exist, as we have seen above) and more in the player's interpretation of the dynamics of play. As such they are examples of procedural rhetoric, systems in which the rules themselves convey meaning (Bogost 2009). A key difference between the approaches lies in the authorial effort required to create works of each kind. In theory, both approaches allow works of arbitrary complexity, but in practice the expressivity of each is bound by the labour involved in design and implementation. The scripted approach is limited by the writer's ability to conceive and write many different pathways; the systemic approach is limited by the designer's ability to create a system with rich play dynamics. The scripted approach offers representational freedom and expressivity. A branching narrative can effectively describe, through written text and pre-rendered animation, whatever scene the writer desires. Likewise, the characters in the game can be as varied and lifelike as the author is able to conjure, because their reactions can be scripted in advance to match the situations the author knows will arise. This freedom comes at a cost. The effort of writing branching narratives 23 grows both with the length and the breadth of the story. As a result, designers have an incentive to limit the amount of branching possible. A common solution is to give the narrative a "string of beads" episodic structure (Salen & Zimmerman 2003, p. 279; Costikyan 2007). Each episode contains a small number of "beats" which branch into different paths, but ultimately dovetail back to an overall linear narrative before proceeding to the next episode. A single beat typically offers the player two to four options for action, explicitly enumerated. Any more than this would tax both the writer's ability to author outcomes and the player's ability to read and choose between them. As a result, the player's agency in such games is openly proscribed. They are offered a small number of paths to take, all of which show the heavy footprints of the author ahead of them. Contrast this with the systemic approach in which the game mechanics more directly represent the procedures of the subject matter; in the case of PP, document checking and simple household economics. The consequences of player action are computed by the application of generalpurpose rules to the current game state, rather than explicitly scripted. Complexity arises from the interaction between rules and the resulting variety in game states (Juul 2002). The authorship burden now lies with the game designer to devise a system that creates interesting play dynamics which tell a meaningful narrative. There is an important distinction to be made in terms of the granularity of choice between these two approaches. A branching-narrative game generally presents the player with a small number of high-impact choices (shoot the dog vs. save the dog). In a systemic game, the actions are smaller, more numerous and often individually inconsequential (move left vs. move right). The 24 meaning comes through the emergent dynamics between the player and the system, what Sicart calls the aggregation of choices (Sicart 2013, p. 105): Branching narratives are often retraceable... The aggregation of choices is a better fit for designing ethical gameplay because it places player in a narrative or world context in which many choices are offered all the time, and the consequence of each is not easily traceable to a particular choice. When game narratives are formed as the aggregation of many small choices, the player has greater scope for nuanced control over the system, and the hand of the author is better concealed. A game's dynamics may be the result of careful design, but to the player they feel emergent rather than planned, and in a complex game there is always the possibility of the player discovering a strategy the designer has not anticipated. This creates a different sense of agency in the player and places more of the authorial burden on the player's shoulders. It also implies that the meaning that emerges for players will partly depend on the choices they make and the approaches to the game that they adopt. Nevertheless, our current ability to create systems of meaningful play cannot rival the stories we can tell in the scripted approach. Personal and social dynamics in particular are difficult to model and the hunt for a playable system of "social physics" is still an open research problem (Khandaker-Kokoris 2015; McCoy et al. 2011, 2013). As a result, systemic gameplay tends to be limited to physical and economic interaction, with relatively anonymous game characters. In multiplayer games this problem can be solved by encouraging interaction between players, but in a single-player game such as PP is it a limiting factor. 25 While we speak of these the systemic and scripted approaches to design as producing different kinds of games, in practice many games employ both approaches in a single title. Computer roleplaying games, for example, often include a systemic implementation of combat, driven by statistics for strength, health, dexterity, etc., alongside a scripted implementation of dialogue and branching narrative. The two approaches rarely combine smoothly however, because of the mismatch in levels of granularity. The fine-grained representation used in the systemic simulation must be squared-off to fit in the coarse boxes of the scripted narrative, and much detail is lost in the transition. In the worst cases this mismatch leads to ludonarrative dissonance (Hocking 2007; Treanor 2013, pp.14-15) in which the scripted and systemic elements of a game offer conflicting interpretations. Morality meters are a good example of this. They are an attempt to take narrative-level actions (giving water to a beggar, blowing up a city) and translate them into a simulation-level statistic (an aggregated karma score), which is then converted back into one of a handful of narrative-level categories (Good, Neutral, Evil) that affect the scripted story. As a result, in Fallout 3 it is possible to make up for mass-murder by piling water bottles on beggars. The system of aggregating karma points is at odds with the narrative significance of the acts performed, creating dissonance. Perhaps as a result of this dichotomy, we observe that most ethically notable games focus on either one approach or the other for creating their ethical impact, and so we shall treat them as distinct in the analysis below. 6. Games, Morals, and Moral Psychology To appreciate the effects that a systemic approach to game design can have on the player's moral engagement, we first need an understanding of the player's moral psychology. How do players (and people in general) make moral decisions? What are the different skills involved in doing so? Much of the discussion around games and ethics has suffered from a lack of detailed engagement 26 with the moral psychology literature and, as a result, there tends to be too much focus on the act of judgement and not enough focus on other ethical skills. Recent research has attempted to overcome this deficiency by turning to the theories of moral psychology to gain a better understanding of players' moral functioning (Christen, Faller et al. 2012, Anonymous 2016a). For most of the twentieth century moral psychology was dominated by the work of Lawrence Kohlberg, who argued that deliberate rational inquiry is the cornerstone of moral judgement and the engine of moral development (Kohlberg et al. 1983, p.69; Kohlberg 1981, p.141). Subsequent research has since called Kohlberg's approach into serious doubt (Lapsley & Narvaez 2005; Krebs & Denton 2006; Vozzola 2014), and currently there are a few promising alternatives vying to replace it as the discipline's dominant paradigm. While these differ significantly in their respective approaches, one thing they have in common is their commitment to the idea that morality is a kind of expertise: a set of social, cognitive, emotional, and perceptual skills. One of the most promising and robust alternatives is the Four Component model proposed by Rest and colleagues (Rest et al. 1999; Narvaez & Vaydich 2008). According to the Four Component model the skills that comprise moral expertise can be divided into four interconnected categories: 1) Moral Focus – the extent to which one is committed to one's moral choices and the degree to which one prioritises moral concerns over others. Experts in Moral Focus cultivate ethical self-regulation and foster an ethical identity that leads them to revere life and deepen a commitment to a moral way of living. 2) Moral Sensitivity– the ability to identify morality in the real world, to understand the motivations of others, and to perceive the consequences of one's behaviour. Experts in 27 moral sensitivity are better at quickly and accurately 'reading' a moral situation and determining what role they might play in it. 3) Moral Judgement – the ability to understand moral concepts and reason about moral issues. Experts in Moral Judgment have many tools for solving complex moral problems. They reason about duty and consequences, responsibility and religious codes. 4) Moral Action – the ability to overcome temptations and persist in the face of adversity. Doing the right thing even when it's hard. Experts in Moral Action know how to keep their "eye on the prize," enabling them to stay on task and take the necessary steps to get the ethical job done. They are able to intervene courageously and take initiative for others. (Narvaez 2006, p.716; Narvaez 2008) Within each of these categories Narvaez and colleagues have identified seven sub-skills (making 28 in total) that can be targeted as part of moral expertise development. For example, moral sensitivity includes skills such as "taking the perspective of others", while moral action emphasises "taking initiative as a leader", "resolving conflict", and "communicating well" (Lapsley & Narvaez 2005, p.156). Engaging any of the four components of moral expertise implies engaging at least one of their associated sub-skills.7 In addition to being what is perhaps the most well-supported and widely accepted account of moral functioning in the history of moral psychology (Vozzola 2014, Chapter 3), the Four Component model is of particular value to game designers because it highlights the diversity of skills that players bring to bear on moral problems. As designers and critics, it is easy to focus 7 The Four Component Model is a holistic account of moral expertise and as such there is significant overlap between each of the above-listed skills as well as their associated sub-skills. Thus, when we write below about PP "engaging" a specific component, such as moral judgment, we mean it engages that component more extensively than other components. 28 narrowly on the challenge of moral judgement in games at the expense of other skills. The model therefore prompts us to ask not only "What is right or wrong?" (moral judgement), but also "How does the player recognise moral situations?" (moral sensitivity), "Why does the player care?" (moral focus), "How does this challenge the player to act?" and "What resistance does the player face?" (moral action). In this way, this taxonomy provides us with a more nuanced set of lenses (in the manner of Schell (2014)) with which to design and critique the moral content of videogames. 6.1 Moral Focus Moral focus is more than just doing the right thing because it is the right thing; it is also the process of developing an ethical identity and acting with integrity. While moral focus was originally the third of Rest's four components, more recent researchers have given it greater prominence as an overarching component motivating all the other skills (Tanner and Christen 2013, Thoma and Bebeau 2013), and indeed it is a major consideration when designing an ethically engaging game. There is a natural inclination for players to regard games as "not real" and "just a game" and therefore dismiss the moral significance of their actions (Heron and Belford 2014). If designers are to encourage morally reflective players of their games, then engaging the player's moral focus is a prerequisite to all other aspects of design. The player must become "complicit" (Sicart 2013) with the game's morality, making moral decisions for moral, not instrumental, reasons. Moral focus is strongly connected to identity. A person may be morally motivated by a personal or professional self-image which emphasises moral integrity (Thoma and Bebeau 2013). PP takes advantage of this by placing the player in a professional role of some authority and 29 responsibility,8 and giving them an opportunity to develop an identity in terms of their loyalty to the state, the individuals they process, the revolution, their family, or their own skin, without forcing the player to explicitly choose between them. On the one hand this is freeing. The game does not prescribe its own morality meter by rating actions as "Good" or "Evil". Any motivation for moral integrity must come from the players themselves, not from the mechanics of the game. On the other hand, it gives players little external feedback on how their moral character is viewed by others, reducing their incentive to develop an explicit ethical identity. This makes having the moral focus to form and act from an ethical identity particularly challenging in PP. This is not, however, a feature of systemic ethical gameplay per se. Fallout: New Vegas (Obsidian Entertainment 2010), for example, supplants the binary Good/Evil karma meter with a system of reputation meters which measure the esteem of various different in-game factions.9 This encourages players to consider their moral identity through the eyes of others, whose approval or disapproval they may value as they choose. The challenge here for the systemic approach is to make players care enough about what others think in the absence of convincing scripting. This is less of a challenge for games that take a scripted approach, since such games have the ability to tailor specific story outcomes with strong characterisation which can more easily encourage a moral focus in players. 8 Although, with characteristic irony, the game deliberately undermines this professional identity by revealing the player as an untrained neophyte and the player's superiors as uncaring and corrupt. 9 The karma system still exists within the game but it is of much less importance than in earlier Fallout titles or other CRPGs. 30 A disadvantage of the systemic approach is the danger of "ethical fading" (Bazerman & Tenbrunsel 2011, pp. 30-31). An emphasis on systems over role-play can lead the player to focus on the problem of optimizing pragmatic outcomes rather than engaging with ethical considerations. The scripted approach, with its strengths in characterization and world-building, can give the player more incentive to role-play a consistent moral identity, while expecting the game to recognise and respond to it. This strength can become a liability, however, when the scripted alternatives misinterpret the player's intent due to the limited paths available in the storyline. 6.2 Moral Sensitivity Moral sensitivity is the ability to read a situation by recognising its morally salient features, understanding one's own role in it, and understanding the perspectives of others. The systemic approach of PP allows the morality of gameplay to arise from the aggregation of many small choices, without the need to explicitly flag morally salient issues. This leaves it completely up to players to recognise the morality implicit in their situation, identifying morally salient affordances and reasoning about the moral implications of their actions on others. This can be a desirable attribute of ethical gameplay, as Sicart (2010, p. 10) describes: "Players are informed about their state in the game, but they are not informed about the ethical meaning of that state...so the player has to actively reconstruct and interpret the meaning of her role as [an] agent in game." This can require greater moral sensitivity on the part of players. This is illustrated in PP by the great difficulty of remaining sensitive, under the game's dehumanising pressures, to the moral dimensions of one's choices. It is because of this difficulty that one can miss the entire 31 "moral story" of PP and play the game in a purely reactive way without even being aware of doing so. The systemic approach of PP can, however, falter when it comes to rich characterisation. The lack of a systemic social model makes it difficult to program characters who react to the nuance of system dynamics. For this reason, characters tend to be flat (such as the travellers in PP) or altogether absent (such as the Inspector's family). This deficiency, combined with the instrumental thinking encouraged by regarding the game as a system to be optimised, helps to encourage the dehumanisation effect we identified in PP. While in PP this effect is an intended part of the overall message, in other titles it could be a major obstacle to a player's moral involvement in the game. This absence of rich characterisation also helps to explain why moral sensitivity is such an important skill in systemic games such as PP, since there is often little or no explicit feedback from other characters or the game's systems about the morality of the player's actions, thereby requiring players to expertly exercise their moral sensitivity to recognise the presence of moral content. In contrast, a scripted approach has much more opportunity to make its characters rich and reactive. More varied, reactive and engaging characters can give the player more to consider when assessing the ethical impact of a game choice. The design also allows the author to create one-on-one conversations with individual non-player characters, thereby encouraging the player to investigate different perspectives on a problem. On the one hand, this provides rich alternative points of view which the player needs to grasp. On the other hand, this takes away some of the moral burden from players, freeing them from the need to imaginatively recreate the perspectives of others, since these perspectives are simply given to them. 32 It is in regards to moral sensitivity that we see the greatest differences between the systemic and scripted approaches. The scripted choices of branching narratives tend to telegraph their moral significance due to their compressed nature. Morality typically boils down to a small number of dramatic choices between strongly divergent alternatives. In such situations the moral issues and possibilities for action are clearly laid out for the player, meaning there is little call for sensitivity to the presence of moral affordances. In systemic games, since moral issues arise dynamically from the game's systems and are not typically made explicit to the player, much greater skill in moral sensitivity can be required. 6.3 Moral Judgement Moral judgment is the ability to judge what one morally ought to do. Moral judgment is the component of ethical expertise most strongly called upon by many games. This is because the content of games are often designed to specifically challenge this skill by posing complex and difficult moral judgment scenarios. However, the need for moral judgment only becomes clear when one is sufficiently morally sensitive to identify that a situation calls for judgment in the first place. Once a player recognises that moral judgment is required, both systemic and scripted approaches can engage with and test a player's judgment to a similar extent. For example, PP requires the player (as the fairness and loyalty themes illustrate) to make complex moral judgments that weigh up the competing claims of the Inspector's family, his job, the state, revolutionary groups, the wider public, his own moral integrity, and the traveller before him in a matter of seconds with limited information. This is no easy thing. Games taking a scripted approach, such as The Walking Dead, can also contain many examples of the player needing to make complex moral judgments in a limited time. 33 Nonetheless, there can be some important differences between the two approaches. While in PP each traveller and the strength of their respective moral claims on the Inspector to bend the rules for them differ in terms of details (from the bribing smuggler to the distressed wife), there remains a great deal of similarity between each case. The choice is almost always the same: to admit, deny or detain. The relevant considerations are often overlapping and similar. The types of moral judgments required of the player are therefore repetitious to a significant degree. Similarly to the issue of limited characterisation, scripted approaches differ from systemic approaches in that they can craft a range of moral judgment scenarios that are limited only by the designer's imagination and the narrative context. Scripted approaches therefore tend to be able to produce greater breadth of moral judgment scenarios. Systemic approaches can, in contrast, focus with greater intensity or depth on a particular moral theme or issue by designing systems, such as those in PP, which require players to make many similar (but contextually different) judgments. In this regard the scripted approach can offer more breadth, the systemic approach more depth. The tendency (discussed above) for scripted games to reduce moral choices to a handful of clearly signposted alternatives also minimises the amount of creative problem solving the player needs to employ in executing their moral judgement; they are only required to evaluate the options given to them, rather than invent their own solution within the boundaries of a game's systems. 6.4 Moral Action 34 Moral action is the ability to follow through and do what you judge is morally best. This might require resisting temptation, as well as courage and perseverance in the face of obstacles. In cases where one's ends impact on or involve the cooperation of others, leadership, resolving conflict and communicating well may also be required subskills. Both systemic and scripted approaches can and do set up cases where there are temptations to act contrary to one's moral judgment. For example, in PP the player may be tempted to wrongly admit a smuggler purely for the sake of a bribe. Morally similar scenarios can arise in games that take a scripted approach. But the variety and type of temptations that the player can be faced with will tend to be greater in scripted approaches. This is due to the limitations of system complexity and the greater scope for variety that a scripted approach can offer. One final point of difference is that the systemic approach can make the implementation of a moral judgment into action more difficult to achieve than in a scripted approach. In PP it can be quite difficult to actually implement one's judgment. For example, even if one judges that it is morally wrong not to detain wanted criminals, one can easily make a mistake and miss spotting a criminal or lack the perseverance needed to check each document carefully enough. In contrast, in a scripted game one usually only has to click on one of a few dialogue options to implement a moral judgment, and the action will be put into effect automatically. There is little need for perseverance in such cases. Scripted Systemic Moral Focus Strong characterisation and worldbuilding can encourage players to roleplay, making choices based on their assumed moral identity and how they are perceived by others. A focus on systems over stories and characters can lead to "ethical fading" and an emphasis on playing to optimise pragmatic outcomes over ethical behaviour. 35 "Reputation systems" can encourage the player to reflect on their identity in the gameworld. Moral Sensitivity Rich and reactive characterisation can test the player's ability to read moral concerns for others. Branching dialogues tend to foreground moral problems and explicitly enumerate solutions, relieving the player of the burden of recognizing situations for themselves. Allows the morality of gameplay to arise from the aggregation of many small choices, leaving it up to the player to recognise the moral significance of a given scenario. A lack of strong systems of "social physics" limits the kinds of ethical problems that can be represented. Poor characterisation can lead to dehumanisation. Moral Judgement Allows designers to craft a broad diversity of moral scenarios within the context of the narrative. Pre-authored solutions limit the player's ability to creatively resolve dilemmas and temptations. Systemic constraints promote repetition of moral dilemmas and temptations, allowing for in-depth exploration of moral themes. Allows players to concoct creative solutions to moral problems within the boundaries of the system. Moral Action Allows for a variety of scripted scenarios and temptations to test the player's resolve. Scripted conversations allow NPCs to directly challenge the player's choices and commitments. Allows designers to make the implementation of moral decisions challenging. This can represent the "doing" part of morality with procedural affordances. Table 1. Summary of differences between scripted and systemic approaches with respect to implementing the four components in gameplay. 7. Conclusion In this paper we have explored the less common systemic approach to designing videogames that are morally engaging through the example of PP. We have demonstrated the ways in which this game's simple mechanics, revolving around document checking, can give rise to numerous 36 complex moral themes such as dehumanisation, privacy, fairness and loyalty. By contrasting the systemic approach of PP with the more common scripted approach, and by introducing and drawing on the Four Component model, we showed the different degrees to which these two approaches engage with different aspects of a player's various ethical expertise. In particular, a systemic approach can more strongly challenge and engage with a player's moral sensitivity and can require greater persistence in terms of moral action. Both approaches can engage with a player's moral judgment and moral focus. But in this regards a scripted approach can provide for greater scope in the variety and types of challenges that can be presented to players, whereas a systemic approach can provide finer grained scope for moral agency, more freedom for players to frame their own moral options, and a more detailed focus at the expense of breadth. Systemic approaches, however, tend to suffer from weak or limited characterisations, which the scripted approach can excel at. We have summarised these differences in Table 1. This show us that these two different approaches to designing videogames will tend to lead to games that engage with player's ethical expertise in distinctively different ways and to different degrees. This conclusion is not only important in its own right, but it also demonstrates the effectiveness of utilising the Four Component model as a way of thinking about the different approaches we can take to designing games that are morally engaging. This conclusion, however, raises further questions about the extent to which videogames, such as PP, can help to build, and not merely engage with, ethical expertise, as well as the relative effectiveness of videogames at doing this in comparison with other methods (Christen 2012; Anonymous 2016b). References Anonymous. (2007). Details omitted for double-blind reviewing. = Formosa, P. (2007). Is radical evil banal? Is banal evil radical?, Philosophy & Social Criticism, 33(6), 717–735. 37 Anonymous. (2010). Details omitted for double-blind reviewing. = Staines, D. (2010). Video games and moral pedagogy; A neo-Kohlbergian approach. In K. Schierer & D. Gibson (Eds.), Ethics and game design (pp. 35–51). Hershey, PA: IGI Global. Anonymous. (2016a). Details omitted for double-blind reviewing. = Staines, D. (2016). Morality play: Designing videogames for moral expertise development. PhD, University of New South Wales, Kensington, Australia. Anonymous. (2016b). Details omitted for double-blind reviewing. = Ryan, M., Staines, D., & Formosa, P. (2016). Four lenses for designing morally engaging games. In Proceedings of the 1st Joint Conference of DiGRA and FDG. Dundee, Scotland. (in press) Adams, E., & Dormans, J. (2012). Game Mechanics. Berkeley: New Riders. Alexander, L. (2013). Designing the bleak genius of Papers, Please. http://www.gamasutra.com/view/news/199383/Designing_the_bleak_genius_of_Papers_Please.p hp. Accessed April 2015. Allen, N. (2014). Bad Karma: A Critique of Morality Systems in Games. http://www.thepalaceofwisdom.co.uk/blog/2014/01/23/badkarmagames/. Accessed 10/12/2015 2015. Arendt, H. (1965). Eichmann in Jerusalem. New York: Penguin. Aristov, M. (2012). An alternative to the Karma Meter. http://blag.koveras.org/2012/05/29/analternative-to-the-karma-meter/. Accessed July 2015. Bazerman, M. H., & Tenbrunsel, A. E. (2011). Blind Spots: Why We Fail to Do What's Right and What to Do about it. Princeton: Princeton University Press. Belman, J., & Flanagan, M. (2010). Designing games to foster empathy. International Journal of Cognitive Technology, 15(1), 11. 38 Bogost, I. (2009). Persuasive Games. Cambridge, MA: MIT Press. Cain, A. (2005). Books and becoming good. Journal of Moral Education, 34(2), 171-183. Carr, D. (2006a). Moral education at the movies. Journal of Moral Education, 35(3), 319-333. Carr, D. (2006b). On the contribution of literature and the arts to the educational cultivation of moral virtue, feeling and emotion. Journal of Moral Education, 35(3), 319–333. Christen, M., et al. (2012). Serious Moral Games. Zürich, Institute for Design Research, Zürich University of Arts. Cogburn, J., & Silcox, M. (Eds.). (2008). Philosophy Through Video Games. New York: Routledge. Costikyan, G. (2007). Games, Storytelling, and Breaking the String. In P. Harrigan, & N. Wardrip-Fruin (Eds.), Second Person (pp. 5-17). Cambridge, MA: MIT Press. Davis, J. (2013). The emotional impact of Papers, Please. GamesBeat Community Posts. http://venturebeat.com/community/2013/08/13/the-emotional-impact-of-papers-please. Accessed July 2016. Eco, U. (1979). The Role of the Reader. Indiana: Indiana University Press. Ernest, A. (2010). Fundamentals of game design (2nd Edition ed.). Berkeley: New Riders. Fine, R. (2010). Dehumanising the dehumanisers. Journal of Global Ethics, 6(2), 179-190. Flanagan, M., Nissenbaum, H., Belman, J., & Diamond, J. (2007) A method for discovering values in digital games. In Situated Play, Proceedings of DiGRA 2007 Conference. Floyd, D. (2010). Video Games and Moral Choices. https://www.youtube.com/watch?v=6_KU3lUx3u0. Accessed December 2015. Frasca, G. (2001). Videogames of the oppressed. Georgia Institute of Technology, Atlanta, GE. Friedman, M. (1991). Reclaiming the sex/gender distinction. Noüs, 25(2), 200-201. 39 Gee, J. P. (2003). What Video Games have to Teach Us about Learning and Literacy. New York, NY: Palgrave Macmillan. Heron, M. J., & Belford, P. H. (2014). Do You Feel Like a Hero Yet? Externalised Morality in Video Games. Journal of Games Criticism, 1(2), 1-22. Hocking, C. (2007). "Ludonarrative Dissonance in Bioshock." Click Nothing. http://clicknothing.typepad.com/click_nothing/2007/10/ludonarrative-d.html. Accessed June 2016. Hunicke, R., LeBlanc, M., & Zubek, R. (2004). MDA: A Formal Approach to Game Design and Game Research. In Proceedings of the AAAI workshop on Challenges in Game AI (Vol. 4): AAAI Press. Juul, J. (2002). The Open and the Closed. In F. Mäyrä (Ed.), Proceedings of the Computer Games and Digital Cultures conference (pp. 323-329). Tampere: Tampere University Press. Kant, I. (1999). Practical Philosophy. Cambridge: Cambridge University Press. Khandaker-Kokoris, M. (2015). Thinking About People: Designing Games for Social Simulation. http://www.gamasutra.com/blogs/MituKhandakerKokoris/20150325/239662/Thinking_About_P eople_Designing_Games_for_Social_Simulation.php. Accessed December 2015. Kohlberg, L. (1981). Essays on moral development: Vol. 1. San Francisco: Harper & Row. Kohlberg, L., Levine, C., & Hewer, A. (1983). Moral stages. Basel: Karger. Koo, G., & Seider, S. (2010). Video games for prosocial learning. In K. Schierer, & D. Gibson (Eds.), Ethics and game design (pp. 16-33). Hershey, PA: IGI Global. Krebs, D. L., & Denton, K. (2006). Explanatory limitations of cognitive-developmental approaches to morality. Psychological Review, 113(3), 672-675. 40 Lapsley, D., & Narvaez, D. (2005). The psychological foundations of everyday morality and moral expertise. In D. Lapsley, & P. C (Eds.), Character psychology and character education (pp. 140-165). Notre Dame, IN: University of Notre Dame Press. McCoy, J., Treanor, M., Samuel, B., Mateas, M., & Wardrip-Fruin, N. (2011). Prom Week: Social Physics as Gameplay. In 6th International Conference on Foundations of Digital Games (pp. 319-321): ACM. McCoy, J., Treanor, M., Samuel, B., Reed, A. A., Mateas, M., & Wardrip-Fruin, N. (2013). Prom Week: Designing Past the Game/Story Dilemma. In 8th International Conference on the Foundations of Digital Games: Society for the Advancement of the Science of Digital Games. Melenson, J. (2010). The Axis of Good and Evil. In K. Schrier (Ed.), Designing Games for Ethics (pp. 57-71). Hershey, PA: IGI Global. Narvaez, D. (2006). Integrative Ethical Education. In M. Killen, & J. Smetana (Eds.), Handbook of Moral Development (pp. 703-732). Mahwah, NJ: Lawrence Erlbaum Associates. Narvaez, D. (2008). Human Flourishing and Moral Development. In L. Nucci, T. Krettenauer, & D. Narvaez (Eds.), Handbook of Moral and Character Education (pp. 310-327). New York, NY: Routledge. Narvaez, D., Mattan, B., MacMichael, C., & Squillace, M. (2008). Kill bandits, collect gold or save the dying: The effects of playing a prosocial video game. Media Psychology Review, 1(1). Narvaez, D., & Vaydich, J. L. (2008). Moral development and behaviour under the spotlight of the neurological sciences. Journal of Moral Education, 37(3), 289-312. Parker, R. (2013). The Art of Papers, Please: Juul's The Art of Failure Meets Lucas Pope's Papers, Please. http://www.firstpersonscholar.com/the-art-of-papers-please/. Accessed July 2015. 41 Rest, J. R., Narvaez, D., M., B., & S., T. (1999). Postconventional moral thinking: A NeoKohlbergian Approach. Mahwah, NJ: Lawrence Erlbaum Assciates. Salen, K., & Zimmerman, E. (2003). Rules of Play. Cambridge, MA: MIT Press. Schell, J. (2014). The Art of Game Design: A book of Lenses. Boca Raton, Florida: CRC Press. Schrier, K. (2015). EPIC: a framework for using video games in ethics education. Journal of Moral Education, 44(4), 33. Short, E. (2015). Tightening the World-Plot Interface: or, Why I Am Obsessed with Conversation Models. https://emshort.wordpress.com/2015/05/24/framed-invisible-parties-andthe-world-plot-interface/. Accessed June 2015. Sicart, M. (2009). The banality of simulated evil: designing ethical gameplay. Ethics and Information Technology, 1(3), 191-202. Sicart, M. (2010). Values between systems. In K. Schrier (Ed.), Ethics and Game Design (pp. 115). Hershey, PA: IGI Global. Sicart, M. (2013). Beyond Choices: The design of ethical gameplay. Cambridge, MA: MIT Press. spupy (2013). Comment on "End of 2013 Discussions – Papers, Please". http://www.reddit.com/r/Games/comments/1sxy6a/end_of_2013_discussions_papers_please/ce3 5tvp. Accessed July 2015. Sitzmann, T. (2011). A Meta-Analytic Examination of the Instructional Effectiveness of Computer-Based Simulation Games. Personnel Psychology, 64, 489-528. Stevenson, J. (2010). A framework for classification and criticism of ethical games. In K. Schrier (Ed.), Designing Games for Ethics. Hershey, PA: IGI Global. Sweetser, P., & Wiles, J. (2005). Scripting versus emergence. International Journal of Intelligent Games and Simulations, 4(1), 1-9. 42 Tanner, C. & Christen, M. (2013). Moral Intelligence – A Framework for Understanding Moral Competences. In M. Christen, C. v. Schaik, J. Fischer, M. Huppenbauer & C. Tanner (Eds.), Empirically Informed Ethics: Morality between Facts and Norms (pp. 119-136). Springer. Thoma, S. J. & Bebeau, M. J. (2013). Moral Motivation and the Four Component Model. In K. Heinrichs, F. Oser and T. Lovat. Rotterdam (Eds.), Handbook of Moral Motivation (pp.49-68), Sense Publishers: Rotterdam. Toma, E. (2015). Self-reflection and morality in critical games. Who is to be blamed for war? Journal of Comparative Research in Anthropology and Sociology, 6(1), 209-224. Vaughn, S. (1990). Morality and entertainment. The Journal of American History, 77(1), 39-65. Vozzola, E. (2014). Moral development. New York: Routledge. Wouters, P. et. al. (2013). A Meta-Analysis of the Cognitive and Motivational Effects of Serious Games. Journal of Educational Psychology, 105(2), 249-265. Yiluoma, J. (2014). Speed Demos Archive: Papers, Please. Speed Demos Archive. http://speeddemosarchive.com/PapersPlease.html. Accessed July 2016. Zagal, J. (2010). Ethical reasoning and reflection as supported by single-player videogames. In Games cited Balance of Power (Crawford, 1985) Fable (series) (Lionhead Studios, 2004-2010) Fallout 3 (Bethesda Game Studios, 2008) Fallout: New Vegas (Obsidian Entertainment, 2010) inFamous (series) (Sucker Punch Productions, 2009-2014) Mass Effect (series) (BioWare, 2007-2012) 43 Papers, Please (Lucas Pope, 2013) Shadow of the Colossus (Team Ico, 2005) Spec Ops: The Line (Yager Development, 2012). Star Wars: Knights of the Old Republic (BioWare, 2003) The Walking Dead (Telltale, 2012) The Witcher 3: The Wild Hunt (CD Projekt Red, 2015) This War of Mine (11-bit Studios, 2014) Ultima IV: Quest of the Avatar (Origin Systems, 1985)