The authors respond to four JME commentaries on their Feature Article, ‘Autonomy-based criticisms of the patient preference predictor’.

The patient preference predictor is a computer-based algorithm devised to predict the medical treatment that decisionally incapacitated patients would have preferred. The target paper argues against various criticisms to the effect that the use of a PPP is inconsistent with proper respect for patient autonomy.1 In this commentary, I aim to add some clarifications to the complex relationship between autonomy and the PPP. First, I highlight one way in which the decision of a surrogate designated by the patient realises respect (...) for patient autonomy in a way that PPP cannot realise by introducing the idea of shared agency. Second, I show how the attempt to predict someone’s hypothetical choice can be disrespectful to her autonomy. Following the treatment decision of a surrogate designated by the patient counts as a way of respecting the patient’s own autonomy. Designating someone as one’s surrogate is a case of entrusting her with the task of making important life decisions on one’s behalf. To entrust someone with a certain matter is to assign her the responsibility for that matter based on the trust that she will take a good care of it. When a patient entrusts a surrogate decision-maker, what is the matter that she is trusted to take a good care of? In many contexts, the surrogate is r egarded as an epistemic proxy, one who is likely to know the patient’s preferences the best. It is often thought that the point of practising surrogate decision making is to respect the patient’s autonomy by following what the patient most likely would have preferred. Using a PPP in the decision-making process has the same goal of making a more accurate prediction of the patient’s treatment preference. However, at least in some cases, designating the patient’s intimates or ones she loves and cares …. (shrink)

The use of machine learning, or “artificial intelligence” (AI) in medicine is widespread and growing. In this paper, I focus on a specific proposed clinical application of AI: using models to predict incapacitated patients’ treatment preferences. Drawing on results from machine learning, I argue this proposal faces a special moral problem. Machine learning researchers owe us assurance on this front before experimental research can proceed. In my conclusion I connect this concern to broader issues in AI safety.

The question of how to treat an incapacitated patient is vexed, both normatively and practically—normatively, because it is not obvious what the relevant objectives are; practically, because even once the relevant objectives are set, it is often difficult to determine which treatment option is best given those objectives. But despite these complications, here is one consideration that is clearly relevant: what a patient prefers. And so any device that could reliably identify a patient’s preferences would be a promising tool for (...) guiding the treatment of incapacitated patients. The patient preference predictor is just such a tool—an algorithm that takes as inputs a patient’s sociodemographic characteristics, and outputs a reliable prediction about that patient’s treatment preferences.1 But some have worried that the use of such a tool would violate or fail to appropriately respect patients’ autonomy. There are, I think, two ways to understand this kind of criticism. First, globally—as a worry that any systematic implementation of the PPP would be problematic on the grounds that it would result in significant or pervasive autonomy violations. Second, locally—as a worry that in some important range of cases, certain uses of the PPP would be problematic on autonomy grounds. Jardas et al, as I read them, address global autonomy-based criticisms, arguing—convincingly, in my view—that there’s no reason to suspect the autonomy concerns raised by the PPP would be so significant and pervasive as to render any implementation of it generally problematic.1 But even with the global criticisms rebuffed, there remains work to be done. Any ethically acceptable implementation of the PPP must be sensitive to the more local autonomy-based criticisms, with restrictions and safeguards in place to ensure respect for autonomy in the kinds of cases in which the use of the PPP might otherwise threaten …. (shrink)

BackgroundHealthcare providers have to make ethically complex clinical decisions which may be a source of stress. Researchers have recently introduced Artificial Intelligence (AI)-based applications to assist in clinical ethical decision-making. However, the use of such tools is controversial. This review aims to provide a comprehensive overview of the reasons given in the academic literature for and against their use.MethodsPubMed, Web of Science, Philpapers.org and Google Scholar were searched for all relevant publications. The resulting set of publications was title and abstract (...) screened according to defined inclusion and exclusion criteria, resulting in 44 papers whose full texts were analysed using the Kuckartz method of qualitative text analysis.ResultsArtificial Intelligence might increase patient autonomy by improving the accuracy of predictions and allowing patients to receive their preferred treatment. It is thought to increase beneficence by providing reliable information, thereby, supporting surrogate decision-making. Some authors fear that reducing ethical decision-making to statistical correlations may limit autonomy. Others argue that AI may not be able to replicate the process of ethical deliberation because it lacks human characteristics. Concerns have been raised about issues of justice, as AI may replicate existing biases in the decision-making process.ConclusionsThe prospective benefits of using AI in clinical ethical decision-making are manifold, but its development and use should be undertaken carefully to avoid ethical pitfalls. Several issues that are central to the discussion of Clinical Decision Support Systems, such as justice, explicability or human–machine interaction, have been neglected in the debate on AI for clinical ethics so far.Trial registrationThis review is registered at Open Science Framework (https://osf.io/wvcs9). (shrink)

When a patient lacks sufficient capacity to make a certain treatment decision, whether because of deficits in their ability to make a judgement that reflects their values or to make a decision that reflects their judgement or both, the decision must be made by a surrogate. Often the best way to respect the patient’s autonomy, in such cases, is for the surrogate to make a ‘substituted’ judgement on behalf of the patient, which is the decision that best reflects the patient’s (...) values and identity-infused beliefs as they were when the patient last possessed capacity. It is by now well-known that there is troubling evidence that surrogates do only slightly better than chance at making accurate substituted judgements. The use of a patient preference predictor, an algorithm that predicts patient preferences based on characteristics like the patient’s age, gender, socioeconomic status and education level, could help. Rid and Wendler have done important work making a moral case for using the PPP.1 If it were shown to enhance accuracy, it is not at all obvious what objections there could be to its use. Yet several subtle criticisms of the PPP have emerged in the literature. In their feature article, Jardas, Wasserman and Wendler select six of the most prominent objections to the PPP in the literature and defend the PPP against them.2 Along the way they make significant contributions to broader issues concerning the proper uses of statistical evidence and limits on the kinds of preferences there can be reasons of autonomy to respect. Since I found the authors’ answers to the final two objections—that the PPP relies on naked statistical evidence and that it uses non-endorsed reasons—to be especially interesting and provocative, I will examine those answers and raise a few questions about …. (shrink)

Artificial intelligence (AI) systems are quickly gaining ground in healthcare and clinical decision-making. However, it is still unclear in what way AI can or should support decision-making that is based on incapacitated patients’ values and goals of care, which often requires input from clinicians and loved ones. Although the use of algorithms to predict patients’ most likely preferred treatment has been discussed in the medical ethics literature, no example has been realised in clinical practice. This is due, arguably, to the (...) lack of a structured approach to the epistemological, ethical and pragmatic challenges arising from the design and use of such algorithms. The present paper offers a new perspective on the problem by suggesting that preference predicting AIs be viewed as sociotechnical systems with distinctive life-cycles. We explore how both known and novel challenges map onto the different stages of development, highlighting interdisciplinary strategies for their resolution. (shrink)

How shall we decide for others who cannot decide for themselves? And who—or what, in the case of artificial intelligence — should make the decision? The present issue of the journal tackles several interrelated topics, many of them having to do with surrogate decision making. For example, the feature article by Jardas et al 1 explores the potential use of artificial intelligence to predict incapacitated patients’ likely treatment preferences based on their sociodemographic characteristics, raising questions about the means by which (...) we come to decide for others. And a clinical ethics round table led by Wilkinson and Pillay2 examines the case of a premature baby on life support whose primary surrogate is herself incapacitated. Together, these examples force us to think more deeply about the meaning and significance of taken-for-granted concepts: respect for autonomy, substituted judgement, best interests. We’ll consider the baby first and then turn to AI. “Baby T” is a critically ill newborn delivered prematurely by emergency caesarean section. The mother had entered into a surrogacy arrangement with a same-sex male couple, the intended parents, who were to take over the baby’s care after birth—just as soon as a formal parental order could be obtained through the court. Until then, the birth mother, who had used her own eggs to conceive Baby T along with sperm from an unidentified donor, would have legal and ethical responsibility to decide about the baby’s care. Unfortunately, she too was in critical condition, having fallen unconscious prior to delivery due to a sudden brain haemorrhage. She remained unconscious, and thus incapacitated, during a crucial period in which time-sensitive decisions about Baby T’s care needed to be made, including whether to continue life support. Given the mother’s incapacity, who should …. (shrink)

When making substituted judgments for incapacitated patients, surrogates often struggle to guess what the patient would want if they had capacity. Surrogates may also agonize over having the (sole) responsibility of making such a determination. To address such concerns, a Patient Preference Predictor (PPP) has been proposed that would use an algorithm to infer the treatment preferences of individual patients from population-level data about the known preferences of people with similar demographic characteristics. However, critics have suggested that even if such (...) a PPP were more accurate, on average, than human surrogates in identifying patient preferences, the proposed algorithm would nevertheless fail to respect the patient’s (former) autonomy since it draws on the ‘wrong’ kind of data: namely, data that are not specific to the individual patient and which therefore may not reflect their actual values, or their reasons for having the preferences they do. Taking such criticisms on board, we here propose a new approach: the Personalized Patient Preference Predictor (P4). The P4 is based on recent advances in machine learning, which allow technologies including large language models to be more cheaply and efficiently ‘fine-tuned’ on person-specific data. The P4, unlike the PPP, would be able to infer an individual patient’s preferences from material (e.g., prior treatment decisions) that is in fact specific to them. Thus, we argue, in addition to being potentially more accurate at the individual level than the previously proposed PPP, the predictions of a P4 would also more directly reflect each patient’s own reasons and values. In this article, we review recent discoveries in artificial intelligence research that suggest a P4 is technically feasible, and argue that, if it is developed and appropriately deployed, it should assuage some of the main autonomy-based concerns of critics of the original PPP. We then consider various objections to our proposal and offer some tentative replies. (shrink)