Strange Loops: Apparent versus Actual Human Involvement in Automated Decision-Making

Berkeley Technology Law Journal 34 (3) (2019)
  Copy   BIBTEX


The era of AI-based decision-making fast approaches, and anxiety is mounting about when, and why, we should keep “humans in the loop” (“HITL”). Thus far, commentary has focused primarily on two questions: whether, and when, keeping humans involved will improve the results of decision-making (making them safer or more accurate), and whether, and when, non-accuracy-related values—legitimacy, dignity, and so forth—are vindicated by the inclusion of humans in decision-making. Here, we take up a related but distinct question, which has eluded the scholarship thus far: does it matter if humans appear to be in the loop of decision-making, independent from whether they actually are? In other words, what is stake in the disjunction between whether humans in fact have ultimate authority over decision-making versus whether humans merely seem, from the outside, to have such authority? Our argument proceeds in four parts. First, we build our formal model, enriching the HITL question to include not only whether humans are actually in the loop of decision-making, but also whether they appear to be so. Second, we describe situations in which the actuality and appearance of HITL align: those that seem to involve human judgment and actually do, and those that seem automated and actually are. Third, we explore instances of misalignment: situations in which systems that seem to involve human judgment actually do not, and situations in which systems that hold themselves out as automated actually rely on humans operating “behind the curtain.” Fourth, we examine the normative issues that result from HITL misalignment, arguing that it challenges individual decision-making about automated systems and complicates collective governance of automation.



External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Social and creative decision making.Carl Martin Allwood & Marcus Selart - 2010 - In Carl Martin Allwood & Marcus Selart (eds.), Decision making: Social and creative dimensions. Springer Media.
Ethical Decision Making: Special or No Different? [REVIEW]Dawn R. Elm & Tara J. Radin - 2012 - Journal of Business Ethics 107 (3):313-329.
Towards a Design Science of Ethical Decision Support.Kieran Mathieson - 2007 - Journal of Business Ethics 76 (3):269-292.
Shared decision-making, gender and new technologies.Kristin Zeiler - 2007 - Medicine, Health Care and Philosophy 10 (3):279-287.


Added to PP

331 (#58,703)

6 months
77 (#55,912)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

Daniel Susser
Cornell University

Citations of this work

No citations found.

Add more citations

References found in this work

No references found.

Add more references