Minds and Machines 28 (4):735-774 (2018)
Authors |
|
Abstract |
Interactions between an intelligent software agent and a human user are ubiquitous in everyday situations such as access to information, entertainment, and purchases. In such interactions, the ISA mediates the user’s access to the content, or controls some other aspect of the user experience, and is not designed to be neutral about outcomes of user choices. Like human users, ISAs are driven by goals, make autonomous decisions, and can learn from experience. Using ideas from bounded rationality, we frame these interactions as instances of an ISA whose reward depends on actions performed by the user. Such agents benefit by steering the user’s behaviour towards outcomes that maximise the ISA’s utility, which may or may not be aligned with that of the user. Video games, news recommendation aggregation engines, and fitness trackers can all be instances of this general case. Our analysis facilitates distinguishing various subcases of interaction, as well as second-order effects that might include the possibility for adaptive interfaces to induce behavioural addiction, and/or change in user belief. We present these types of interaction within a conceptual framework, and review current examples of persuasive technologies and the issues that arise from their use. We argue that the nature of the feedback commonly used by learning agents to update their models and subsequent decisions could steer the behaviour of human users away from what benefits them, and in a direction that can undermine autonomy and cause further disparity between actions and goals as exemplified by addictive and compulsive behaviour. We discuss some of the ethical, social and legal implications of this technology and argue that it can sometimes exploit and reinforce weaknesses in human beings.
|
Keywords | Artificial intelligence Autonomy Machine learning Human–computer interaction Nudge Persuasion Recommender System Intelligent System Ethics |
Categories | (categorize this paper) |
Reprint years | 2018 |
ISBN(s) | |
DOI | 10.1007/s11023-018-9479-0 |
Options |
![]() ![]() ![]() ![]() |
Download options
References found in this work BETA
Theory of Games and Economic Behavior.John Von Neumann & Oskar Morgenstern - 1944 - Princeton, NJ, USA: Princeton University Press.
View all 40 references / Add more references
Citations of this work BETA
Recommender Systems and Their Ethical Challenges.Silvia Milano, Mariarosaria Taddeo & Luciano Floridi - 2020 - AI and Society (4):957-967.
The Ethics of Digital Well-Being: A Thematic Review.Christopher Burr, Mariarosaria Taddeo & Luciano Floridi - 2020 - Science and Engineering Ethics 26:2313–2343.
Can Machines Read Our Minds?Christopher Burr & Nello Cristianini - 2019 - Minds and Machines 29 (3):461-494.
Varieties of Artifacts: Embodied, Perceptual, Cognitive, and Affective.Richard Heersmink - 2021 - Topics in Cognitive Science (4):1-24.
The Ethics of Digital Well-Being: A Thematic Review.Christopher Burr, Mariarosaria Taddeo & Luciano Floridi - 2020 - Science and Engineering Ethics 26 (4):2313–2343.
View all 26 citations / Add more citations
Similar books and articles
Anonymity and Software Agents: An Interdisciplinary Challenge. [REVIEW]Frances Brazier, Anja Oskamp, Corien Prins, Maurice Schellekens & Niek Wijngaards - 2004 - Artificial Intelligence and Law 12 (1-2):137-157.
A User Profiling Component with the Aid of User Ontologies.Istvan-Tibor Nébel - 2003 - In Proceedings of Learning - Teaching - Knowledge - Adaptivity. University of Karlsruhe.
A User Profiling Component with the Aid of User Ontologies.Nébel István-Tibor, Barry Smith & Paschke Ralf - 2003 - In Learning – Teaching – Knowledge – Adaptivity (LLWA), University of Karlsruhe (2003). Karlsruhe, Germany:
From Participatory Design to Participating Problem Solving: Enhancing System Adaptability Through User Modelling. [REVIEW]Zhengxin Chen - 1993 - AI and Society 7 (3):238-247.
Reader as User: Applying Interface Design Techniques to the Web.Karen McGrane Chauss - 1996 - Kairos: A Journal of Rhetoric, Technology, and Pedagogy 1 (2).
Intelligent Agents and Liability: Is It a Doctrinal Problem or Merely a Problem of Explanation? [REVIEW]Emad Abdel Rahim Dahiyat - 2010 - Artificial Intelligence and Law 18 (1):103-121.
Towards an Artificial User: The “What” Problem for an Architecture Capable of Developing New Goals.Vincenzo Tagliasco - unknown
Relationships Between Obligations and Actions in the Context of Institutional Agents, Human Agents or Software Agents.Robert Demolombe - 2011 - Artificial Intelligence and Law 19 (2-3):99-115.
Emergent Values for Automatons: Ethical Problems of Life in the Generalized Internet.Eric Steinhart - 1999 - Ethics and Information Technology 1 (2):155-160.
Cognitive Automata and the Law: Electronic Contracting and the Intentionality of Software Agents. [REVIEW]Giovanni Sartor - 2009 - Artificial Intelligence and Law 17 (4):253-290.
Interaction and Resistance: The Recognition of Intentions in New Human-Computer Interaction.Vincent C. Müller - 2011 - In Anna Esposito, Antonietta M. Esposito, Raffaele Martone, Vincent C. Müller & Gaetano Scarpetta (eds.), Towards autonomous, adaptive, and context-aware multimodal interfaces: Theoretical and practical issues. Springer. pp. 1-7.
The Role of Cognitive Modeling for User Interface Design Representations: An Epistemological Analysis of Knowledge Engineering in the Context of Human-Computer Interaction. [REVIEW]Markus F. Peschl & Chris Stary - 1998 - Minds and Machines 8 (2):203-236.
Spyware – the Ethics of Covert Software.Mathias Klang - 2004 - Ethics and Information Technology 6 (3):193-202.
Analytics
Added to PP index
2018-09-25
Total views
169 ( #70,469 of 2,519,864 )
Recent downloads (6 months)
23 ( #37,758 of 2,519,864 )
2018-09-25
Total views
169 ( #70,469 of 2,519,864 )
Recent downloads (6 months)
23 ( #37,758 of 2,519,864 )
How can I increase my downloads?
Downloads