Generative AI enables automated, effective manipulation at scale. Despite the growing general ethical discussion around generative AI, the specific manipulation risks remain inadequately investigated. This article outlines essential inquiries encompassing conceptual, empirical, and design dimensions of manipulation, pivotal for comprehending and curbing manipulation risks. By highlighting these questions, the article underscores the necessity of an appropriate conceptualisation of manipulation to ensure the responsible development of Generative AI technologies.

Many philosophical accounts of manipulation are blind to the extent to which actual people fall short of the rational ideal, while prominent accounts in political science are under-inclusive. We offer necessary and sufficient conditions – Suitable Reason and Testimonial Honesty – distinguishing manipulative from non-manipulative influence; develop a ‘hypothetical disclosure test’ to measure the degree of manipulation; and provide further criteria to assess and compare the morality of manipulation across cases. We discuss multiple examples drawn from politics and from public (...) policy with particular attention to recent debates about the ethics and politics of nudge. (shrink)

This paper rejects an argument defending the view that the boundary between deception and manipulation is such that some manipulations intended to cause false beliefs count as non-deceptive. On the strongest version of this argument, if a specific behaviour involves compromising the victim’s reasoning, then the behaviour is manipulative but not deceptive, and if it involves exposing the victim to misleading evidence that justifies her false belief, then it is deceptive but not manipulative. This argument has been consistently used as (...) a reason to reject the traditional analysis of human deception, according to which intentionally causing someone to acquire a false belief is sufficient for deception. And because the traditional analysis is also consistent with our most basic intuitions about deception, it does matter whether this argument succeeds. (shrink)

In this article, we apply the literature on the ethics of choice-architecture (nudges) to the realm of virtual reality (VR) to point out ethical problems with using VR for empathy-based nudging. Specifically, we argue that VR simulations aiming to enhance empathic understanding of others via perspective-taking will almost always be unethical to develop or deploy. We argue that VR-based empathy enhancement not only faces traditional ethical concerns about nudge (autonomy, welfare, transparency), but also a variant of the semantic variance problem (...) that arises for intersectional perspective-taking. VR empathy simulations deceive and manipulate their users about their experiences. Despite their often laudable goals, such simulations confront significant ethical challenges. In light of these goals and challenges, we conclude by proposing that VR designers shift from designing simulations aimed at producing empathic perspective-taking to designing simulations aimed at generating sympathy. These simulations, we claim, can avoid the most serious ethical issues associated with VR nudges, semantic variance, and intersectionality. (shrink)

In this article, we argue that rational persuasion can be a _pro tanto_ wrong and that online platforms possess features that are especially conducive to this wrong. We begin by setting out an account of rational persuasion. This consists of four jointly sufficient conditions for rational persuasion and is intended to capture the core, uncontroversial cases of such persuasion. We then discuss a series of wrong-making features which are present in methods of influence commonly thought of as _pro tanto_ wrong, (...) such as manipulation and paternalism. It is next shown that these wrong-making features are also present in a range of cases that are, by the jointly sufficient conditions already established, rational persuasion, and so some forms of rational persuasion are _pro tanto_ wrong, including in some ways that have not previously been remarked upon. Finally, we demonstrate that online settings possess a number of features that are especially conducive to wrongful rational persuasion. (shrink)

A series of recent papers raises worries about the manipulative potential of algorithmic transparency (to wit, making visible the factors that influence an algorithm’s output). But while the concern is apt and relevant, it is based on a fraught understanding of manipulation. Therefore, this paper draws attention to the ‘indifference view’ of manipulation, which explains better than the ‘vulnerability view’ why algorithmic transparency has manipulative potential. The paper also raises pertinent research questions for future studies of manipulation in the context (...) of algorithmic transparency. (shrink)

The permissibility of nudging in public policy is often assessed in terms of the conditions of transparency, rationality, and easy resistibility. This debate has produced important resources for any ethical inquiry into nudging, but it has also failed to focus sufficiently on a different yet very important question, namely: when do nudges undermine a patient’s voluntary consent to a medical procedure? In this paper, I take on this further question and, more precisely, I ask to which extent the three conditions (...) of transparency, rationality, and easy resistibility can be applied to the assessment of voluntary consent too. After presenting two examples, designed to put pressure on these three conditions, I show that, suitably modified, the three conditions can remain significant in the assessment of voluntary consent as well. However, the needed modifications are very substantial and result in a rather complicated view. To propose a tidier solution, I argue that nudging undermines voluntary consent if and only if it cannot be ‘interpersonally justified’ to the patient. I use the three modified conditions to motivate the idea of interpersonal justification and also to further specify the principles it involves. My resulting view is especially attractive because it builds on already existing insights from the debate on nudging, updates those insights with an eye to medical consent, and finally unites them in an elegant and simple framework. (shrink)