Abstract

The existential risk posed by ubiquitous artificial intelligence (AI) is a subject of frequent discussion with descriptions of the prospect of misuse, the fear of mass destruction, and the singularity. In this paper I address an under-explored category of existential risk posed by AI, namely emotional risk. Values are a main source of emotions. By challenging some of our most essential values, AI systems are therefore likely to expose us to emotional risks such as loss of care and loss of meaning. Part one presents a study of a leading bank in Germany where an AI system was implemented to replace humans in decision-making processes. Part two explains why humans actively make use of values to make decisions. Part three shows the connection between values and emotions. Part four relates parts two and three to the bank study by giving concrete examples of how the employees saw their roles as workers, and how the relationships to their customers changed emotionally after the AI system was implemented to make decisions.