Skip to main content
SpringerLink
Log in

Artificial moral agents: moral mentors or sensible tools?

  • Original Paper
  • Published:
Ethics and Information Technology Aims and scope Submit manuscript

Abstract

The aim of this paper is to offer an analysis of the notion of artificial moral agent (AMA) and of its impact on human beings’ self-understanding as moral agents. Firstly, I introduce the topic by presenting what I call the Continuity Approach. Its main claim holds that AMAs and human moral agents exhibit no significant qualitative difference and, therefore, should be considered homogeneous entities. Secondly, I focus on the consequences this approach leads to. In order to do this I take into consideration the work of Bostrom and Dietrich, who have radically assumed this viewpoint and thoroughly explored its implications. Thirdly, I present an alternative approach to AMAs—the Discontinuity Approach—which underscores an essential difference between human moral agents and AMAs by tackling the matter from another angle. In this section I concentrate on the work of Johnson and Bryson and I highlight the link between their claims and Heidegger’s and Jonas’s suggestions concerning the relationship between human beings and technological products. In conclusion I argue that, although the Continuity Approach turns out to be a necessary postulate to the machine ethics project, the Discontinuity Approach highlights a relevant distinction between AMAs and human moral agents. On this account, the Discontinuity Approach generates a clearer understanding of what AMAs are, of how we should face the moral issues they pose, and, finally, of the difference that separates machine ethics from moral philosophy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. This is why the authors spend many words explaining the level of abstraction (LoA) methodology, which allows them to achieve a particular outcome—the inclusion of AMAs in the category of moral agents—without at the same time reducing all moral agents to the basically technological model they resort to. In fact, it is possible to acknowledge the status of moral agents to machines or computer software only if a particular LoA is assumed. That is, the assumption of a specific LoA sets the conditions by which a particular claim makes sense. If these conditions change, then the same claim must be reassessed at a different LoA. Although highly controversial, the recourse to similar methodological precautions is in my opinion of utmost importance in the field of machine ethics, since it helps avoiding epistemological trespasses which lead to confusion and conceptual fuzziness.

  2. In this paper, by “machine ethics” I mean the technological discipline the purpose of which is to design, build, and develop robots exhibiting some sort of moral behaviour—not the philosophical discussion about how such technological undertaking impinges on our traditional moral ideas. This helps to keep the technological perspective separate from the philosophical one, which is a very important distinction not to be forgotten. For similar concerns regarding the expression “machine ethics” see Anderson (2011), who distinguishes between machine ethics and machine metaethics; and Torrance (2011), who distinguishes between practical and philosophical machine ethics.

  3. In distinguishing between an operative approach and, as it were, a “philosophical” or “definitional” approach to moral agency I follow Gunkel 2012, p. 74. Gunkel stresses the difference between, on the one hand, Floridi’s and Sanders’s “functional approach” to moral agency, which implies “engineering solutions” or “‘effective characterization(s)’ that can work” and, on the other hand, the discussion of the “fundamental philosophical problem” of moral agency, which entails “advancing and defending a decision concerning a definition”. Although the issue lurks at the back of this distinction, I do not intend to take any stance on the epistemological opposition between realism and constructivism. The only point I wish to highlight here is that operative approaches organize knowledge in accordance with a specific purpose they assume in advance and intend to achieve—in the case of machine ethics, the technological reproduction of human moral behaviour. On the contrary, “philosophical” approaches like those I have in mind here strive to understand things independently from any productive purpose—i.e., absolutely—although of course this may very well lead to practically relevant conclusions. This argument is based in the well-known Aristotelian distinction between theoria and poiesis.

  4. A thorough assessment of the technological model of moral agency as a methodological assumption would require a completely different analysis, which is beyond the purposes of this paper. On this, see Wallach and Allen (2009, §§5–11) and Gunkel ( 2012, pp. 74–87).

  5. Similarly, Hall claims that future “superethical machines” (Hall 2011a, p. 42) will render society a better place for human beings too, which makes the effort towards the manufacturing of conscious machines a moral duty.

  6. To some extent, a similar role can be acknowledged to science fiction too. Sure enough, philosophical discussions and fictional writings are very different forms of human expression, which aim at different targets through different means. However, a very interesting intercourse between the two has always been occurring, since literary imagination can offer radical insight into the implication of concepts, which may have already found their way to common sense. In the field of machine ethics the most interesting example is that of Isaac Asimov’s well-known collection of stories I, Robot, where the author proposed his Three (or One plus Three) Laws of Robotics—a set of laws intended to serve as a moral guideline for future technological agents. The laws have been seriously taken into account since the problem of implementing moral constraints in machines passed from fictional imagination to engineering practice (Clarke 2011).

  7. In philosophy of technology the instrumental approach is often criticized on the account that it supposedly supports the thesis that technological tools are neutral and transparent entities, which are simply used and by no means feedback on the users’ practices and relations to the world (Verbeek 2005; Kiran and Verbeek 2010). I cannot go deep into this now, but it must be said that neither Heidegger nor Jonas endorsed such a claim. In fact, they both submitted an opposite interpretation: tools, as simple or as advanced as they may be, do transform practices and contribute to shaping the human experience of the world by both modifying how ends are achieved and making new ends achievable, as Verbeek (2005) correctly points out. Even so, they remain tools, i.e., objects we primarily resort to in order to achieve our ends. So, in my opinion, it is not necessary to dismiss the instrumental approach to technology altogether. What is necessary, however, is to avoid any “externalist” oversimplification in defining what tools are and how they impact on human existence.

  8. In a very similar way Jonas criticised Wiener’s cybernetics in his essay Cybernetics and purpose (Jonas 1953). To some extent, even the well-known paper by Searle (1980) may be interpreted along this line. McDermott (2008) and Beavers (2012) propose similar arguments in the field of machine ethics.

  9. It is to be briefly noticed that, although learning machines may be thought of as self-determining entities, they still are built in order to serve some purpose set by human beings, as all machines are. So, their form of “self-determination”, being still of a functional kind, does not match the way human beings set purposes and values. In fact, machine “self-determination” is always embedded in practical contexts the general purposes and values of which are already set by human beings.

  10. This tendency, however, is of course enforced by human-likeness, so that to a certain degree it is possible to exploit it in order to promote the social acceptability of artificial agents (Mori 1970; Duffy 2003, 2013; Fink 2012). Moreover, if machines are to be deployed in human contexts, they must be able to interact with objects which have been designed for human use, like stairs, doorknobs, or trays, and with us. So, machines inevitably feature cues which trigger our inclination to frame them in human terms.

References

  • Allen, C., Varner, G., & Zinser, J. (2000). Prolegomena to any future artificial moral agent. Journal of Experimental and Theoretical Artificial Intelligent, 12, 251–261.

    Article  MATH  Google Scholar 

  • Anderson, S. L. (2011). Machine metaethics. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 21–27). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Beavers, A. F. (2012). Moral machines and the threat of ethical nihilism. In P. Lin, K. Abney & G. A. Bekey (Eds.), Robot ethics. The ethical and social implications of robotics (pp. 333–344). Cambridge: The M.I.T. Press.

    Google Scholar 

  • Bostrom, N. (2003). Ethical issues in advanced artificial intelligence. https://nickbostrom.com/ethics/ai.html. Accessed 22 Aug 2017.

  • Bostrom, N. (2014). Superintelligence. Paths, dangers, strategies. Oxford: Oxford University Press.

    Google Scholar 

  • Bryson, J. J. (2010). Robots Should Be Slaves. In Y. Wilks (Ed.), Close engagements with artificial companions: Key social, psychological, ethical and design issues (pp. 63–74). Amsterdam: John Benjamins.

    Chapter  Google Scholar 

  • Bryson, J. J., & Kime, P. (2011). Just an artifact: Why machines are perceived as moral agents. https://www.cs.bath.ac.uk/~jjb/ftp/BrysonKime-IJCAI11.pdf. Accessed 22 Aug 2017.

  • Clarke, R. (2011). Asimov’s laws of robotics. Implications for information technology. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 254–284). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Dennett, D. C. (1997). When HAL kills, who’s to blame? Computer ethics. In D. G. Stork (Ed.), Hal’s legacy: 2001’s computer as dream and reality (pp. 351–366). Cambridge: The M.I.T. Press.

    Google Scholar 

  • Dietrich, E. (2007). After humans are gone. Journal of Experimental and Theoretical Artificial Intelligence, 19(1), 55–67.

    Article  Google Scholar 

  • Dietrich, E. (2011). Homo Sapiens 2.0. Building the better robots of our nature. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 531–538). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Duffy, B. (2003). Anthropomorphism and the social robot. Robotic and Autonomous Systems, 42, 177–190.

    Article  MATH  Google Scholar 

  • Duffy, B. (2013). Anthropomorphism and robotics. http://medialabeurope.org/anthropos/publications/pubsIAISB02-Duffy.pdf. Accessed 28 Nov 2017.

  • Fabris, A. (2016). Philosophy, image and the mirror of machines. In Ž. Paić & K. Purgar (Eds.), Theorizing images (pp. 111–120). Newcastle upon Tyne: Cambridge Scholars.

    Google Scholar 

  • Fink, J. (2012). Anthropomorphism and human likeness in the design of robots and human-robot interaction. In S. S. Ge et al. (Eds.), ICSR 2012, LNAI 7621, pp. 199–208.

  • Floridi, L., & Sanders, J. W. (2004). On the morality of artificial agents. Minds and Machine, 14, 349–379.

    Article  Google Scholar 

  • Franklin, S., & Graesser, A. (1996). Is it an agent, or just a program? A taxonomy for autonomous agents. In J. P. Müller, M. J. Wooldridge & N. R. Jennings (Eds.), Intelligent Agents III. Agent Theories, Architectures, and Languages. ATAL 1996. Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence), vol. 1193 (pp. 22–35). Berlin: Springer.

    Google Scholar 

  • Friedman, B., & Kahn, P. H. (1992). Human agency and responsible computing: Implications for computer system design. Journal of Systems Software, 17(7), 7–14.

    Article  Google Scholar 

  • Fussel, S. R., Kiesler, S., Setlock, L. D., & Yew, V. (2008). How people anthropomorphize robots. In HRI’08 Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction (pp 145–152).

  • Gips, J. (1995). Towards the ethical robot. In G. K. Ford, C. Glymour & P. J. Hayes (Eds.), Android epistemology (pp. 243–252). Cambridge: The M.I.T. Press.

    Google Scholar 

  • Grodzinsky, F. S., Miller, K. W., & Wolf, M. J. (2008). The ethics of designing artificial agents. Ethics and Information Technology, 10, 115–121.

    Article  Google Scholar 

  • Gunkel, D. J. (2012). The machine question. Critical perspectives on AI, robots and ethics. Cambridge: The M.I.T. Press.

    Google Scholar 

  • Hall, J. S. (2011a). Ethics for machines. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 28–44). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Hall, J. S. (2011b). Ethics for self-improving machines. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 512–523). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Heidegger, M. (2010). Being and time. New York: State University of New York Press.

    Google Scholar 

  • Heidegger, M. (2013). The question concerning technology and other essays. New York: Harper Perennial.

    Google Scholar 

  • Henry, B. (2014). Imaginaries of the Global Age. “Golem and others” in the post-human condition. Politica e Società, 2/2014, 221–246.

    Google Scholar 

  • Himma, K. E. (2009). Artificial agency, consciousness, and the criteria for moral agency: What properties must an artificial agent have to be a moral agent? Ethics and Information Technology, 11(1), 19–29.

    Article  Google Scholar 

  • Johnson, D. G. (2003). Computer ethics. In R. G. Frey & C. H. Wellman (Eds.), A companion to applied ethics (pp. 608–619). Malden-Oxford-Carlton: Blackwell.

    Google Scholar 

  • Johnson, D. G. (2011). Computer systems. Moral entities, but not moral agents. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 168–183). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Jonas, H. (1953). Cybernetics and purpose: A critique. Social research, XX(2), pp. 172–192. Reprinted as § 5 in Id. (2001). The Phenomenon of Life. Toward a Philosophical Biology (pp. 108–127). Evanston: Northwestern University Press.

    Google Scholar 

  • Jonas, H. (1959). The practical uses of theory. Social research, XXVI(2), pp. 151–166. Reprinted as § 8 in Id. (2001). The Phenomenon of Life. Toward a Philosophical Biology (pp. 188–210). Evanston: Northwestern University Press.

    Google Scholar 

  • Kakoudaki, D. (2014). Anatomy of a robot. Literature, cinema, and the cultural work of artificial people. New Brunswick: Rutgers University Press.

    Google Scholar 

  • Kiran, A. E., & Verbeek, P.-P. (2010). Trusting our selves to technology. Knowledge, Technology, and Policy, 23, 409–427.

    Article  Google Scholar 

  • Kurzweil, R. (2005). The singularity is near. When Humans transcend biology. New York: Viking.

    Google Scholar 

  • Laukyte, M. (2017). Artificial agents among us. Should we recognize them as agents proper? Ethics and Information Technology, 19(1), 1–17.

    Article  Google Scholar 

  • Lemaignan, S., Fink, J., & Dillenbourg, P. (2014). The Dynamics of Anthropomorphism in Robotics. In HRI’14 Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot Interaction (pp. 226–227).

  • McDermott, D. (2008). What matters to a machine? In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 88–114). Cambridge: Cambridge University Press.

    Google Scholar 

  • Moor, J. H. (1995). Is ethics computable? Metaphilosophy, 26(1–2), 1–21.

    Article  Google Scholar 

  • Moor, J. H. (2006). The nature, importance, and difficulty of machine ethics. IEEE Intelligent Systems, 21(4), 18–21.

    Article  Google Scholar 

  • Moore, G. E. (1965). Cramming more components into integrated circuits. Electronics, 38(8), 114–117.

    Google Scholar 

  • Mori, M. (1970). Bukimi no tani. Energy, 7, 33–35. English version: The Uncanny Valley. IEEE Robotics and Automation Magazine, June 2012, 98–100.

    Google Scholar 

  • Nass, C., & Moon, Y. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56(1), 81–103.

    Article  Google Scholar 

  • Nissenbaum, H. (2001). How computer systems embody values. Computer, 34, 118–120.

    Article  Google Scholar 

  • Scheutz, M. (2012). The inherent dangers of unidirectional emotional bonds between humans and social robots. In P. Lin, K. Abney & G. A. Bekey (Eds.), Robot ethics. The ethical and social implications of robotics (pp. 205–222). Cambridge: The MIT Press.

    Google Scholar 

  • Searle, J. R. (1980). Minds, brains, and programs. The Behavioral and Brain Sciences, 3, 417–424.

    Article  Google Scholar 

  • Sullins, J. P. (2011). When is a robot a moral agent? In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 151–161). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Torrance, S. (2011). Machine ethics and the Idea of a more-than-human moral world. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 115–137). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Turing, A. M. (1950). Computing machinery and intelligence. Mind, LIX(236), 433–460.

    Article  MathSciNet  Google Scholar 

  • Turkle, S. (2011). Authenticity in the age of digital companions. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 62–76). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Verbeek, P.-P. (2005). What Things Do. Philosophical Reflections on Technology, Agency, and Design. University Park: The Pennsylvania State University Press.

    Google Scholar 

  • Vinge, V. (1993). The coming technological singularity: How to survive in the post-human era. Vision-21: Interdisciplinary Science and Engineering in the Era of Cyberspace (pp. 11–22). NASA Scientific and Technical Information Program.

  • Wallach, W. (2010). Robot minds and human ethics: the need for a comprehensive model of decision making. Ethics and Information Technology, 12(3), 243–250.

    Article  Google Scholar 

  • Wallach, W., & Allen, C. (2009). Moral machines. Teaching robots right from wrong. New York: Oxford University Press.

    Book  Google Scholar 

  • Wallach, W., Allen, C., & Smit, I. (2011). Why machine ethics? In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 51–61). Cambridge: Cambridge University Press.

    Google Scholar 

  • Whitby, B. (2011). On computable morality: An examination of machines as moral advisors. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 138–150). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Yudkowsky, E. (2008). Artificial intelligence as a positive and negative factor in global risk. Machine Intelligence Research Institute. http://intelligence.org/files/AIPosNegFactor.pdf. Accessed online 22 Aug 2017.

Download references

Author information

Authors and Affiliations

  1. Institute of Law, Politics and Development, Sant’Anna School of Advanced Studies, Pisa, Italy

    Fabio Fossa

Authors
  1. Fabio Fossa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fabio Fossa.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fossa, F. Artificial moral agents: moral mentors or sensible tools?. Ethics Inf Technol 20, 115–126 (2018). https://doi.org/10.1007/s10676-018-9451-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10676-018-9451-y

Keywords

Search

Navigation