Skip to main content

Advertisement

SpringerLink
Log in

Artificial Moral Agents: A Survey of the Current Status

  • Review
  • Published:
Science and Engineering Ethics Aims and scope Submit manuscript

Abstract

One of the objectives in the field of artificial intelligence for some decades has been the development of artificial agents capable of coexisting in harmony with people and other systems. The computing research community has made efforts to design artificial agents capable of doing tasks the way people do, tasks requiring cognitive mechanisms such as planning, decision-making, and learning. The application domains of such software agents are evident nowadays. Humans are experiencing the inclusion of artificial agents in their environment as unmanned vehicles, intelligent houses, and humanoid robots capable of caring for people. In this context, research in the field of machine ethics has become more than a hot topic. Machine ethics focuses on developing ethical mechanisms for artificial agents to be capable of engaging in moral behavior. However, there are still crucial challenges in the development of truly Artificial Moral Agents. This paper aims to show the current status of Artificial Moral Agents by analyzing models proposed over the past two decades. As a result of this review, a taxonomy to classify Artificial Moral Agents according to the strategies and criteria used to deal with ethical problems is proposed. The presented review aims to illustrate (1) the complexity of designing and developing ethical mechanisms for this type of agent, and (2) that there is a long way to go (from a technological perspective) before this type of artificial agent can replace human judgment in difficult, surprising or ambiguous moral situations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Abbass, H. A., Petraki, E., Merrick, K., Harvey, J., & Barlow, M. (2016). Trusted autonomy and cognitive cyber symbiosis: Open challenges. Cognitive Computation,8(3), 385–408.

    Article  Google Scholar 

  • Alaieri, F., & Vellino, A. (2016). Ethical decision making in robots: Autonomy, trust and responsibility. In International conference on social robotics (pp. 159–168). Cham: Springer.

  • Allen, C., Smit, I., & Wallach, W. (2005). Artificial morality: Top-down, bottom-up, and hybrid approaches. Ethics and Information Technology,7(3), 149–155.

    Article  Google Scholar 

  • Amstutz, M. R. (2013). International ethics: Concepts, theories, and cases in global politics. New York: Rowman & Littlefield Publishers.

    Google Scholar 

  • Anderson, M., & Anderson, S. L. (2007a). Machine ethics: Creating an ethical intelligent agent. AI Magazine,28(4), 15–26.

    Google Scholar 

  • Anderson, M., & Anderson, S. L. (2007b). The status of machine ethics: A report from the AAAI symposium. Minds and Machines,17(1), 1–10.

    Article  Google Scholar 

  • Anderson, M., & Anderson, S. L. (2008). Ethical healthcare agents. In M. Sordo, S. Vaidya, & L. C. Jain (Eds.), Advanced computational intelligence paradigms in healthcare-3 (pp. 233–257). Berlin: Springer.

    Chapter  Google Scholar 

  • Anderson, M., & Anderson, S. L. (2010). Robot be good. Scientific American,303(4), 72–77.

    Article  Google Scholar 

  • Anderson, M., & Anderson, S. L. (2014). Geneth: A general ethical dilemma analyzer. In Twenty-eighth AAAI conference on artificial intelligence (pp. 253–261).

  • Anderson, M., Anderson, S. L., & Armen, C. (2004). Towards machine ethics. In Proceedings of the AOTP’04—The AAAI-04 workshop on agent organizations: Theory and practice.

  • Anderson, M., Anderson, S. L., & Armen, C. (2005). Medethex: Toward a medical ethics advisor. In Proceedings of the AAAI 2005 Fall symposium on caring machines: AI in elder care (pp. 9–16).

  • Anderson, M., Anderson, S. L., & Armen, C. (2006a). An approach to computing ethics. IEEE Intelligent Systems,21(4), 56–63.

    Article  Google Scholar 

  • Anderson, M., Anderson, S. L., & Armen, C. (2006b). Medethex: A prototype medical ethics advisor. In Proceedings of the national conference on artificial intelligence (Vol. 21, No. 2, pp. 1759–1765). Menlo Park, CA/Cambridge, MA: AAAI Press/MIT Press.

  • Andino, C. (2015). Place of ethics between technical knowledge. A philosophical approach. Revista Científica de la UCSA,2(2), 85–94.

    Google Scholar 

  • Arkin, R. (2009). Governing lethal behavior in autonomous robots. London: Chapman and Hall/CRC.

    Book  Google Scholar 

  • Arkin, R. C. (2010). The case for ethical autonomy in unmanned systems. Journal of Military Ethics,9(4), 332–341.

    Article  Google Scholar 

  • Arkin, R. (2018). Lethal autonomous systems and the plight of the noncombatant. In R. Kiggins (Ed.), The political economy of robots (pp. 317–326). Cham: Springer.

    Chapter  Google Scholar 

  • Arkoudas, K., Bringsjord, S., & Bello, P. (2005). Toward ethical robots via mechanized deontic logic. In AAAI Fall symposium on machine ethics (pp. 17–23).

  • Ashrafian, H. (2015). Artificial intelligence and robot responsibilities: Innovating beyond rights. Science and Engineering Ethics,21(2), 317–326.

    Article  Google Scholar 

  • Bandyopadhyay, D., & Sen, J. (2011). Internet of things: Applications and challenges in technology and standardization. Wireless Personal Communications,58(1), 49–69.

    Article  Google Scholar 

  • Batty, M., Axhausen, K. W., Giannotti, F., Pozdnoukhov, A., Bazzani, A., Wachowicz, M., et al. (2012). Smart cities of the future. The European Physical Journal Special Topics,214(1), 481–518.

    Article  Google Scholar 

  • Beauvisage, T. (2009). Computer usage in daily life. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 575–584). ACM.

  • Bedaf, S., Draper, H., Gelderblom, G. J., Sorell, T., & de Witte, L. (2016). Can a service robot which supports independent living of older people disobey a command? The views of older people, informal carers and professional caregivers on the acceptability of robots. International Journal of Social Robotics,8(3), 409–420.

    Article  Google Scholar 

  • Belloni, A., Berger, A., Besson, V., Boissier, O., Bonnet, G., Bourgne, G., et al. (2014). Towards a framework to deal with ethical conflicts in autonomous agents and multi-agent systems. In CEPE 2014 well-being, flourishing, and ICTs (pp. 1–10).

  • Belloni, A., Berger, A., Boissier, O., Bonnet, G., Bourgne, G., Chardel, P. A., et al. (2015). Dealing with ethical conflicts in autonomous agents and multi-agent systems. In 1st International workshop on artificial intelligence and ethics at the 29th AAAI conference on artificial intelligence.

  • Blass, J. A. (2016). Interactive learning and analogical chaining for moral and commonsense reasoning. In Thirtieth AAAI conference on artificial intelligence (pp. 4289–4290).

  • Blass, J. A., & Forbus, K. D. (2015). Moral decision-making by analogy: Generalizations versus exemplars. In Twenty-ninth AAAI conference on artificial intelligence (pp. 501–507).

  • Bonnemains, V., Saurel, C., & Tessier, C. (2018). Embedded ethics: Some technical and ethical challenges. Ethics and Information Technology,20(1), 41–58.

    Article  Google Scholar 

  • Borenstein, J., & Arkin, R. (2019). Robots, ethics, and intimacy: The need for scientific research. In D. Berkich, & M. d'Alfonso (Eds.), On the cognitive, ethical, and scientific dimensions of artificial intelligence (pp. 299–309). Cham: Springer.

    Chapter  Google Scholar 

  • Borst, J. P., & Anderson, J. R. (2015). Using the ACT-R cognitive architecture in combination with fMRI data. In B. Forstmann, & E. J. Wagenmakers (Eds.), An introduction to model-based cognitive neuroscience (pp. 339–352). Berlin: Springer.

    Google Scholar 

  • Brachman, R. J. (2002). Systems that know what they’re doing. IEEE Intelligent Systems,17(6), 67–71.

    Article  Google Scholar 

  • Briggs, G., & Scheutz, M. (2015). Sorry, I can’t do that”: Developing mechanisms to appropriately reject directives in human–robot interactions. In 2015 AAAI Fall symposium series (pp. 1–5).

  • Bringsjord, S., Sundar, G. N., Thero, D., & Si, M. (2014). Akratic robots and the computational logic thereof. In Proceedings of the IEEE 2014 international symposium on ethics in engineering, science, and technology (pp. 1–8). IEEE Press.

  • Brundage, M. (2014). Limitations and risks of machine ethics. Journal of Experimental & Theoretical Artificial Intelligence,26(3), 355–372.

    Article  Google Scholar 

  • Capraro, V., & Rand, D. G. (2018). Do the right thing: Experimental evidence that preferences for moral behavior, rather than equity or efficiency per se, drive human prosociality. Forthcoming in Judgment and Decision Making,13(1), 99–111.

    Google Scholar 

  • Cervantes, J. A., Rodríguez, L. F., López, S., Ramos, F., & Robles, F. (2016). Autonomous agents and ethical decision-making. Cognitive Computation,8(2), 278–296.

    Article  Google Scholar 

  • Cervantes, J. A., Rosales, J. H., López, S., Ramos, F., & Ramos, M. (2017). Integrating a cognitive computational model of planning and decision-making considering affective information. Cognitive Systems Research,44, 10–39.

    Article  Google Scholar 

  • Choi, D., & Langley, P. (2018). Evolution of the icarus cognitive architecture. Cognitive Systems Research,48, 25–38.

    Article  Google Scholar 

  • Coeckelbergh, M. (2010). Moral appearances: Emotions, robots, and human morality. Ethics and Information Technology,12(3), 235–241.

    Article  Google Scholar 

  • Conway, P., & Gawronski, B. (2013). Deontological and utilitarian inclinations in moral decision making: A process dissociation approach. Journal of Personality and Social Psychology,104(2), 216–235.

    Article  Google Scholar 

  • Cook, D. J., & Das, S. K. (2012). Pervasive computing at scale: Transforming the state of the art. Pervasive and Mobile Computing,8(1), 22–35.

    Article  Google Scholar 

  • Cristani, M., & Burato, E. (2009). Approximate solutions of moral dilemmas in multiple agent system. Knowledge and Information Systems,18(2), 157–181.

    Article  Google Scholar 

  • Czubenko, M., Kowalczuk, Z., & Ordys, A. (2015). Autonomous driver based on an intelligent system of decision-making. Cognitive Computation,7(5), 569–581.

    Article  Google Scholar 

  • Dehghani, M., Tomai, E., Forbus, K. D., & Klenk, M. (2008). An integrated reasoning approach to moral decision-making. In Twenty-third AAAI conference on artificial intelligence (pp. 1280–1286).

  • Deng, B. (2015). Machine ethics: The robot’s dilemma. Nature,523(7558), 24–26.

    Article  Google Scholar 

  • Dennis, L. A., Fisher, M., Lincoln, N. K., Lisitsa, A., & Veres, S. M. (2016a). Practical verification of decision-making in agent-based autonomous systems. Automated Software Engineering,23(3), 305–359.

    Article  Google Scholar 

  • Dennis, L., Fisher, M., Slavkovik, M., & Webster, M. (2016b). Formal verification of ethical choices in autonomous systems. Robotics and Autonomous Systems,77, 1–14.

    Article  Google Scholar 

  • Epting, S. (2016). A different trolley problem: The limits of environmental justice and the promise of complex moral assessments for transportation infrastructure. Science and Engineering Ethics,22(6), 1781–1795.

    Article  Google Scholar 

  • Erdur, M. (2018). Moral realism and the incompletability of morality. The Journal of Value Inquiry,52(2), 227–237.

    Article  Google Scholar 

  • Fagin, R., Halpern, J. Y., & Vardi, M. Y. (1990). A nonstandard approach to the logical omniscience problem. In Proceedings of the 3rd conference on theoretical aspects of reasoning about knowledge (pp. 41–55). Morgan Kaufmann Publishers Inc.

  • Feil-Seifer, D., & Matarić, M. J. (2011). Socially assistive robotics. IEEE Robotics and Automation Magazine,18(1), 24–31.

    Article  Google Scholar 

  • Ferrell, O. C., & Gresham, L. G. (1985). A contingency framework for understanding ethical decision making in marketing. The Journal of Marketing,49(3), 87–96.

    Article  Google Scholar 

  • Fleetwood, J., Vaught, W., Feldman, D., Gracely, E., Kassutto, Z., & Novack, D. (2000). Medethex online: A computer-based learning program in medical ethics and communication skills. Teaching and Learning in Medicine,12(2), 96–104.

    Article  Google Scholar 

  • Fumagalli, M., & Priori, A. (2012). Functional and clinical neuroanatomy of morality. Brain,135(7), 2006–2021.

    Article  Google Scholar 

  • Gerdes, A., & Øhrstrøm, P. (2015). Issues in robot ethics seen through the lens of a moral turing test. Journal of Information, Communication and Ethics in Society,13(2), 98–109.

    Article  Google Scholar 

  • Gogoll, J., & Müller, J. F. (2017). Autonomous cars: In favor of a mandatory ethics setting. Science and Engineering Ethics,23(3), 681–700.

    Article  Google Scholar 

  • Govindarajulu, N. S., Bringjsord, S., & Ghosh, R. (2018). One formalization of virtue ethics via learning. arXiv preprint arXiv:180507797.

  • Greene, J. D., Morelli, S. A., Lowenberg, K., Nystrom, L. E., & Cohen, J. D. (2008). Cognitive load selectively interferes with utilitarian moral judgment. Cognition,107(3), 1144–1154.

    Article  Google Scholar 

  • Greene, J., Rossi, F., Tasioulas, J., Venable, K. B., & Williams, B. C. (2016). Embedding ethical principles in collective decision support systems. In Thirtieth AAAI conference on artificial intelligence (pp. 4147–4151).

  • Greene, J. D., Sommerville, R. B., Nystrom, L. E., Darley, J. M., & Cohen, J. D. (2001). An fMRI investigation of emotional engagement in moral judgment. Science,293(5537), 2105–2108.

    Article  Google Scholar 

  • Guerini, M., Pianesi, F., & Stock, O. (2015). Is it morally acceptable for a system to lie to persuade me? In Workshops at the twenty-ninth AAAI conference on artificial intelligence (pp. 53–60).

  • Han, T. A., & Pereira, L. M. (2018). Evolutionary machine ethics. In O. Bendel (Ed.), Handbuch Maschinenethik (pp. 1–25). Wiesbaden: Springer.

    Google Scholar 

  • Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017). Neuroscience-inspired artificial intelligence. Neuron,95(2), 245–258.

    Article  Google Scholar 

  • Honarvar, A. R., & Ghasem-Aghaee, N. (2009). Casuist BDI-agent: A new extended BDI architecture with the capability of ethical reasoning. In International conference on artificial intelligence and computational intelligence (pp. 86–95). Berlin: Springer.

  • Howard, D., & Muntean, I. (2016). A minimalist model of the artificial autonomous moral agent (AAMA). In The 2016 AAAI Spring symposium series (pp. 217–225).

  • Hughes, G. J. (2001). Routledge philosophy guidebook to Aristotle on ethics. London: Routledge.

    Google Scholar 

  • Kahn, P. H., Jr., Kanda, T., Ishiguro, H., Gill, B. T., Ruckert, J. H., Shen, S., et al. (2012). Do people hold a humanoid robot morally accountable for the harm it causes? In Proceedings of the seventh annual ACM/IEEE international conference on Human–Robot Interaction (pp. 33–40). ACM.

  • Kirchin, S. (Ed.). (2012). What is metaethics? In Metaethics (pp. 1–20). London: Palgrave Macmillan.

  • Kishi, T., Hashimoto, K., & Takanishi, A. (2017). Human like face and head mechanism. In A. Goswami, & P. Vadakkepat (Eds.), Humanoid robotics: A reference (pp. 1–26). Dordrecht: Springer.

    Google Scholar 

  • Kruglanski, A. W., & Gigerenzer, G. (2011). Intuitive and deliberate judgments are based on common principles. Psychological Review,118(1), 97–109.

    Article  Google Scholar 

  • Laird, J. E. (2008). Extending the soar cognitive architecture. Frontiers in Artificial Intelligence and Applications,171, 224–235.

    Google Scholar 

  • Laird, J. E., Kinkade, K. R., Mohan, S., & Xu, J. Z. (2012). Cognitive robotics using the soar cognitive architecture. In Workshops at the twenty-sixth AAAI conference on artificial intelligence (pp. 46–54).

  • Laird, J. E., Lebiere, C., & Rosenbloom, P. S. (2017). A standard model of the mind: Toward a common computational framework across artificial intelligence, cognitive science, neuroscience, and robotics. AI Magazine,38(4), 13–26.

    Article  Google Scholar 

  • Lombrozo, T. (2009). The role of moral commitments in moral judgment. Cognitive Science,33(2), 273–286.

    Article  Google Scholar 

  • Long, L. N., & Kelley, T. D. (2010). Review of consciousness and the possibility of conscious robots. Journal of Aerospace Computing, Information, and Communication,7(2), 68–84.

    Article  Google Scholar 

  • Madl, T., & Franklin, S. (2015). Constrained incrementalist moral decision making for a biologically inspired cognitive architecture. In R. Trappl (Ed.), A construction manual for robots’ ethical systems (pp. 137–153). Cham: Springer.

    Chapter  Google Scholar 

  • Malle, B. F. (2016). Integrating robot ethics and machine morality: The study and design of moral competence in robots. Ethics and Information Technology,18(4), 243–256.

    Article  Google Scholar 

  • Malle, B. F., Scheutz, M., Arnold, T., Voiklis, J., & Cusimano, C. (2015). Sacrifice one for the good of many? People apply different moral norms to human and robot agents. In Proceedings of the tenth annual ACM/IEEE international conference on human–robot interaction (pp. 117–124). ACM.

  • Mermet, B., & Simon, G. (2016). Formal verification of ethical properties in multiagent systems. In ECAI 2016 workshop on ethics in the design of intelligent agents (EDIA’16). The Netherlands: The Hague.

  • Metta, G., Natale, L., Nori, F., Sandini, G., Vernon, D., Fadiga, L., et al. (2010). The iCub humanoid robot: An open-systems platform for research in cognitive development. Neural Networks,23(8), 1125–1134.

    Article  Google Scholar 

  • Mikhail, J. (2007). Universal moral grammar: Theory, evidence and the future. Trends in Cognitive Sciences,11(4), 143–152.

    Article  Google Scholar 

  • Moor, J. H. (2006). The nature, importance, and difficulty of machine ethics. IEEE Intelligent Systems,21(4), 18–21.

    Article  Google Scholar 

  • Mordoch, E., Osterreicher, A., Guse, L., Roger, K., & Thompson, G. (2013). Use of social commitment robots in the care of elderly people with dementia: A literature review. Maturitas,74(1), 14–20.

    Article  Google Scholar 

  • Mostafa, S. A., Ahmad, M. S., & Mustapha, A. (2019). Adjustable autonomy: A systematic literature review. Artificial Intelligence Review,51(2), 149–186.

    Article  Google Scholar 

  • Mostafa, S. A., Mustapha, A., Mohammed, M. A., Ahmad, M. S., & Mahmoud, M. A. (2018). A fuzzy logic control in adjustable autonomy of a multi-agent system for an automated elderly movement monitoring application. International Journal of Medical Informatics,112, 173–184.

    Article  Google Scholar 

  • Pellizzoni, S., Siegal, M., & Surian, L. (2010). The contact principle and utilitarian moral judgments in young children. Developmental Science,13(2), 265–270.

    Article  Google Scholar 

  • Podschwadek, F. (2017). Do androids dream of normative endorsement? On the fallibility of artificial moral agents. Artificial Intelligence and Law,25(3), 325–339.

    Article  Google Scholar 

  • Reig, S., Norman, S., Morales, C. G., Das, S., Steinfeld, A., & Forlizzi, J. (2018). A field study of pedestrians and autonomous vehicles. In Proceedings of the 10th international conference on automotive user interfaces and interactive vehicular applications (pp. 198–209). ACM.

  • Rodríguez, L. F., & Ramos, F. (2014). Development of computational models of emotions for autonomous agents: A review. Cognitive Computation,6(3), 351–375.

    Article  Google Scholar 

  • Schaich Borg, J., Hynes, C., Van Horn, J., Grafton, S., & Sinnott-Armstrong, W. (2006). Consequences, action, and intention as factors in moral judgments: An fMRI investigation. Journal of Cognitive Neuroscience,18(5), 803–817.

    Article  Google Scholar 

  • Scheutz, M., & Malle, B. F. (2014). Think and do the right thing: A plea for morally competent autonomous robots. In Proceedings of the IEEE 2014 international symposium on ethics in engineering, science, and technology (p. 9). IEEE Press.

  • Schroeder, M. (2017). Normative ethics and metaethics. In T. McPherson, & D. Plunkett (Eds.), The Routledge handbook of metaethics (pp. 674–686). London: Routledge.

    Chapter  Google Scholar 

  • Sharkey, A., & Sharkey, N. (2012). Granny and the robots: Ethical issues in robot care for the elderly. Ethics and Information Technology,14(1), 27–40.

    Article  Google Scholar 

  • Shigemi, S. (2018). ASIMO and humanoid robot research at Honda. In A. Goswami, & P. Vadakkepat (Eds.), Humanoid robotics: A reference (pp. 1–36). Springer.

  • Tikhanoff, V., Cangelosi, A., & Metta, G. (2011). Integration of speech and action in humanoid robots: iCub simulation experiments. IEEE Transactions on Autonomous Mental Development,3(1), 17–29.

    Article  Google Scholar 

  • Trafton, G., Hiatt, L., Harrison, A., Tamborello, F., Khemlani, S., & Schultz, A. (2013). ACT-R/E: An embodied cognitive architecture for human–robot interaction. Journal of Human–Robot Interaction,2(1), 30–55.

    Article  Google Scholar 

  • Van Riemsdijk, M. B., Jonker, C.M., & Lesser, V. (2015). Creating socially adaptive electronic partners: Interaction, reasoning and ethical challenges. In Proceedings of the 2015 international conference on autonomous agents and multiagent systems (pp. 1201–1206). International Foundation for Autonomous Agents and Multiagent Systems.

  • Van Staveren, I. (2007). Beyond utilitarianism and deontology: Ethics in economics. Review of Political Economy,19(1), 21–35.

    Article  Google Scholar 

  • Van Wynsberghe, A., & Robbins, S. (2018). Critiquing the reasons for making artificial moral agents. Science and Engineering Ethics,25(3), 1–17.

    Google Scholar 

  • Vanderelst, D., & Winfield, A. (2018). An architecture for ethical robots inspired by the simulation theory of cognition. Cognitive Systems Research,48, 56–66.

    Article  Google Scholar 

  • Vernon, D., Metta, G., & Sandini, G. (2007). A survey of artificial cognitive systems: Implications for the autonomous development of mental capabilities in computational agents. IEEE Transactions on Evolutionary Computation,11(2), 151–180.

    Article  Google Scholar 

  • Viroli, M., Pianini, D., Montagna, S., & Stevenson, G. (2012). Pervasive ecosystems: A coordination model based on semantic chemistry. In Proceedings of the 27th annual ACM symposium on applied computing (pp. 295–302). ACM.

  • Von der Pfordten, D. (2012). Five elements of normative ethics—A general theory of normative individualism. Ethical Theory and Moral Practice,15(4), 449–471.

    Article  Google Scholar 

  • Von Wright, G. H. (1951). Deontic logic. Mind,60(237), 1–15.

    Article  Google Scholar 

  • Waldrop, M. M. (2015). Autonomous vehicles: No drivers required. Nature News,518(7537), 20.

    Article  Google Scholar 

  • Walker, L. J., & Hennig, K. H. (2004). Differing conceptions of moral exemplarity: Just, brave, and caring. Journal of Personality and Social Psychology,86(4), 629–647.

    Article  Google Scholar 

  • Wallach, W. (2008). Implementing moral decision making faculties in computers and robots. AI & Society,22(4), 463–475.

    Article  Google Scholar 

  • Wallach, W. (2010). Robot minds and human ethics: The need for a comprehensive model of moral decision making. Ethics and Information Technology,12(3), 243–250.

    Article  Google Scholar 

  • Wallach, W., Allen, C., & Smit, I. (2008). Machine morality: Bottom-up and top-down approaches for modelling human moral faculties. AI & Society,22(4), 565–582.

    Article  Google Scholar 

  • Wallach, W., Franklin, S., & Allen, C. (2010). A conceptual and computational model of moral decision making in human and artificial agents. Topics in Cognitive Science,2(3), 454–485.

    Article  Google Scholar 

  • Wang, S., Wan, J., Zhang, D., Li, D., & Zhang, C. (2016). Towards smart factory for industry 4.0: A self-organized multi-agent system with big data based feedback and coordination. Computer Networks,101, 158–168.

    Article  Google Scholar 

  • Wellman, M. P., & Rajan, U. (2017). Ethical issues for autonomous trading agents. Minds and Machines,27(4), 609–624.

    Article  Google Scholar 

  • Winfield, A. F., Blum, C., & Liu, W. (2014). Towards an ethical robot: Internal models, consequences and ethical action selection. In Conference towards autonomous robotic systems (pp. 85–96). Cham: Springer.

  • Yampolskiy, R. V. (2013). Artificial intelligence safety engineering: Why machine ethics is a wrong approach. In V. Müller (Ed.), Philosophy and theory of artificial intelligence (pp. 389–396). Berlin: Springer.

    Chapter  Google Scholar 

  • Young, L., & Durwin, A. (2013). Moral realism as moral motivation: The impact of meta-ethics on everyday decision-making. Journal of Experimental Social Psychology,49(2), 302–306.

    Article  Google Scholar 

  • Zambonelli, F., & Viroli, M. (2011). A survey on nature-inspired metaphors for pervasive service ecosystems. International Journal of Pervasive Computing and Communications,7(3), 186–204.

    Article  Google Scholar 

  • Zieba, S., Polet, P., Vanderhaegen, F., & Debernard, S. (2010). Principles of adjustable autonomy: A framework for resilient human–machine cooperation. Cognition, Technology & Work,12(3), 193–203.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Centro Universitario de los Valles, Universidad de Guadalajara, Carretera Guadalajara – Ameca Km. 45.5, 46600, Ameca, Mexico

    José-Antonio Cervantes, Sonia López & Salvador Cervantes

  2. Department of Computer Science, Instituto Tecnológico de Sonora, Sonora, Mexico

    Luis-Felipe Rodríguez

  3. Department of Electronics, Systems and Informatics, Instituto Tecnológico y de Estudios Superiores de Occidente, Tlaquepaque, Mexico

    Francisco Cervantes

  4. Department of Computer Science, Centro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional, Guadalajara, Mexico

    Félix Ramos

Authors
  1. José-Antonio Cervantes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sonia López
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luis-Felipe Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Salvador Cervantes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Francisco Cervantes
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Félix Ramos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José-Antonio Cervantes.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cervantes, JA., López, S., Rodríguez, LF. et al. Artificial Moral Agents: A Survey of the Current Status. Sci Eng Ethics 26, 501–532 (2020). https://doi.org/10.1007/s11948-019-00151-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11948-019-00151-x

Keywords

Search

Navigation