Abstract
Ethical morality is one of the significant issues in self-driving cars. The paper provides a newer approach to solve the ethical decision problems in self-driving cars until there is no concrete ethical decision to all problems. This paper gives a two-way approach to solve a problem, with first being the mapping of problem to the solution already known or which has a fixed set of solutions and action priorities defined to a problem previously. Now, if no solution is found or mapping is unsuccessful, then the second stage activates, where the solution from Deep Q-learning model is calculated. It estimates the best Q value and returns that solution or action which maximizes the reward at that instance. The reward function is designed with decreasing priorities and acts accordingly, where the users can change or define their priorities if needed. The case study and results show that the solution that is present in the paper will lead to solving ethical morality problems in self-driving cars up to a great extent.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Awad E, Dsouza S, Kim R, Schulz J, Henrich J, Shariff A, Bonnefon J-F and Rahwan I (2018) The moral machine experiment. Nature 563(7729):59–64. https://doi.org/10.1038/s41586-018-0637-6. [Source : https://www.moralmachine.net/]
Bonnefon J-F, Shariff A, Rahwan I (2016) The Social dilemma of autonomous vehicles. Science. https://doi.org/10.1126/science.aaf2654
Bougie N, Ichise R (2021) Fast and slow curiosity for high-level exploration in reinforcement learning. Appl Intell 51:1086–1107. https://doi.org/10.1007/s10489-020-01849-3
Diuk C, Cohen A and Littman ML (2008) An object-oriented representation for efficient reinforcement learning. In: Proceedings of the 25th international conference on Machine learning, pp 240–247. ACM
Dosovitskiy A, Ros G, Codevilla F, Lopez A and Koltun V (2017) CARLA: an open urban driving simulator. In: Conference on robot learning (pp 1–16). PMLR
Ethically aligned design (2018) A vision for prioritizing human well-being with autonomous and intelligent systems. http://standards.ieee.org/develop/indconn/ec/autonomous_systems.html
Ethik-Kommission (2017) Automatisiertes und vernetztes fahren. Tech Rep. http://www.bmvi.de/berichtethikkommission
European Parliament (2016) Ethical aspects of cyber-physical systems. Scientific Foresight Study. https://goo.gl/Fp5Wjs
Fridman L, Terwilliger J and Jenik B (2018) Deeptraffic: Crowdsourced hyperparameter tuning of deep reinforcement learning systems for multi-agent dense traffic navigation. arXiv preprint. arXiv:1801.02805
Holstein T, Dodig-Crnkovic G (2018) Avoiding the intrinsic unfairness of the trolley problem. IEEE/ACM Int Workshop Softw Fairness (FairWare) 2018:32–37. https://doi.org/10.23919/FAIRWARE.2018.8452918
Holstein T, Dodig-Crnkovic G, Pelliccione P (2018) Ethical and social aspects of self-driving cars. ARXIV’18, January 2018, Gothenburg, Sweden
Kaelbling LP, Littman ML and Moore AW (1996) Reinforcement learning: a survey. CoRR, cs.AI/9605103
Karnouskos S (2020) Self-driving car acceptance and the role of ethics. IEEE Trans Eng Manage 67(2):252–265. https://doi.org/10.1109/TEM.2018.2877307
Kiran BR, Sobh I, Talpaert V, Mannion P, Sallab AAA, Yogamani SK and Pérez P (2020) Deep reinforcement learning for autonomous driving: a survey. CoRR, abs/2002.00444. https://arxiv.org/abs/2002.00444
Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D and Wierstra D (2015) Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971
Maas AL, Hannun AY, Ng AY (2013) Rectifier Nonlinearities improve neural network acoustic models. ICML
Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D and Riedmiller M (2013) Playing atari with deep reinforcement learning. arXiv preprint. arXiv:1312.5602
Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, GOstrovski et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533
NHTSA (2015) Critical reasons for crashes investigated in the national motor vehicle crash causation survey. Tech Rep. https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812115
Riedmiller M, Gabel T, Hafner R, Lange S (2009) Reinforcement learning for robot soccer. Auton Robot 27(1):55–73
Schaul T, Quan J, Antonoglou I and Silver D (2015) Prioritized experience replay. arXiv preprint arXiv:1511.05952
Shalev-Shwartz S, Shammah S and Shashua A (2016) Safe, multi-agent, reinforcement learning for autonomous driving. arXiv preprint. arXiv:1610.03295
Siam M, Elkerdawy S, Jagersand M and Yogamani S (2017) Deep semantic segmentation for automated driving: taxonomy, roadmap and challenges. In: 2017 IEEE 20th international conference on intelligent transportation systems (ITSC) (pp 1–8). IEEE
Sutton RS (1992) Introduction: the challenge of reinforcement learning. In: Sutton RS (ed) Reinforcement learning. The springer international series in engineering and computer science (knowledge representation, learning and expert systems), vol 173. Springer, Boston. https://doi.org/10.1007/978-1-4615-3618-5_1
Sutton RS, Barto AG (2015) Reinforcement learning: an introduction. A Bradford book. The MIT Press Cambridge, Cambridge
Taylor ME and Stone P (2009) Transfer learning for reinforcement learning domains: a survey. J Mach Learn Res 10(56):1633–1685. http://jmlr.org/papers/v10/taylor09a.html
Tesauro G (1995) Temporal difference learning and td-gammon. Commun ACM 38(3):58–68
Todorov E, Erez T and Tassa Y (2012) Mujoco: a physics engine for model-based control. In: Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pages 5026–5033. IEEE
Urmson C, Whittaker WR (2008) Self-driving cars and the urban challenge. IEEE Intell Syst 23(2):66–68
Valdes-Dapena P (2016) Volvo promises death-proof cars by 2020. http://money.cnn.com/2016/01/20/luxury/volvo-no-death-crash-cars-2020
van Hasselt H, Guez A and Silver D (2016a) Deep reinforcement learning with double Q-learning. In: Proceedings of the AAAI Conference on Artificial Intelligence 30(1). Available at: https://ojs.aaai.org/index.php/AAAI/article/view/10295 (Accessed: 18 Mar 2022)
Van Hasselt H, Guez A and Silver D (2016b) Deep reinforcement learning with double q-learning. In AAAI, pp 2094–2100
Wang, P. and Chan, C.Y., 2017, October. Formulation of deep reinforcement learning architecture toward autonomous driving for on-ramp merge. In: 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) (pp 1–6). IEEE
Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8(3–4):279–292
Zhang Y, Huang M and Hao D (2010) The four-level valuation model of ethical decision-making. In: 2010 2nd IEEE International Conference on Information and Financial Engineering, 20 to 10, pp 806–809. https://doi.org/10.1109/ICIFE.2010.5609478
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chandak, A., Aote, S., Menghal, A. et al. Two-stage approach to solve ethical morality problem in self-driving cars. AI & Soc 39, 693–703 (2024). https://doi.org/10.1007/s00146-022-01517-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00146-022-01517-9