The Debate on the Ethics of AI in Health Care: a Reconstruction and Critical Review Jessica Morley1* **, Caio C. V. Machado1*, Christopher Burr1, Josh Cowls1,2, Indra Joshi4, Mariarosaria Taddeo1,2,3, Luciano Floridi1,2,3 1 Oxford Internet Institute, University of Oxford, 1 St Giles', Oxford, OX1 3JS, UK 2 Alan Turing Institute, British Library, 96 Euston Rd, London NW1 2DB, UK 3 Department of Computer Science, University of Oxford, 15 Parks Rd, Oxford, OX1 3QD, UK * These authors contributed equally to the writing of this paper ** Email of correspondence author: Jessica.morley@kellogg.ox.ac.uk Abstract Healthcare systems across the globe are struggling with increasing costs and worsening outcomes. This presents those responsible for overseeing healthcare with a challenge. Increasingly, policymakers, politicians, clinical entrepreneurs and computer and data scientists argue that a key part of the solution will be 'Artificial Intelligence' (AI) – particularly Machine Learning (ML). This argument stems not from the belief that all healthcare needs will soon be taken care of by "robot doctors." Instead, it is an argument that rests on the classic counterfactual definition of AI as an umbrella term for a range of techniques that can be used to make machines complete tasks in a way that would be considered intelligent were they to be completed by a human. Automation of this nature could offer great opportunities for the improvement of healthcare services and ultimately patients' health by significantly improving human clinical capabilities in diagnosis, drug discovery, epidemiology, personalised medicine, and operational efficiency. However, if these AI solutions are to be embedded in clinical practice, then at least three issues need to be considered: the technical possibilities and limitations; the ethical, regulatory and legal framework; and the governance framework. In this article, we report on the results of a systematic analysis designed to provide a clear overview of the second of these elements: the ethical, regulatory and legal framework. We find that ethical issues arise at six levels of abstraction (individual, interpersonal, group, institutional, sectoral, and societal) and can be categorised as epistemic, normative, or overarching. We conclude by stressing how important it is that the ethical challenges raised by implementing AI in healthcare settings are tackled proactively rather than reactively and map the key considerations for policymakers to each of the ethical concerns highlighted. Keywords Artificial Intelligence; Ethics; Healthcare; Health Policies; Machine Learning. 2 1. Introduction Healthcare systems across the globe are struggling with increasing costs and worsening outcomes (Topol, 2019). This presents those responsible for overseeing healthcare systems with a 'wicked problem', meaning that the problem has multiple causes, is hard to understand and define, and hence will have to be tackled from multiple different angles. Against this background, policymakers, politicians, clinical entrepreneurs and computer and data scientists increasingly argue that a key part of the solution will be 'Artificial Intelligence' (AI), particularly Machine Learning (Chin-Yee & Upshur, 2019). The argument stems not from the belief that all healthcare needs will soon be taken care of by "robot doctors" (Chin-Yee & Upshur, 2019). Instead, the argument rests on the classic counterfactual definition of AI as an umbrella term for a range of techniques (summarised in Figure 1 below) that can be used to make machines complete tasks in a way that would be considered intelligent were they to be completed by a human. For example1, as mapped by (Harerimana, Jang, Kim, & Park, 2018), decision tree techniques can be used to diagnose breast cancer tumours (Kuo, Chang, Chen, & Lee, 2001); Support Vector Machine techniques can be used to classify genes (Brown et al., 2000) and diagnose Diabetes Mellitus (Barakat, Bradley, & Barakat, 2010); ensemble learning methods can predict outcomes for cancer patients (Kourou, Exarchos, Exarchos, Karamouzis, & Fotiadis, 2015); and neural networks can be used to recognise human movement (Jiang & Yin, 2015). From this perspective, AI represents a growing resource of interactive, autonomous, and often self-learning (in the machine learning sense, see Figure 1) agency, that can be used on demand (Floridi, 2019), presenting the opportunity for potentially transformative cooperation between machines and doctors (Bartoletti, 2019). 1 For a full overview of all supervised and unsupervised Machine Learning techniques and their applications in healthcare, see Harerimana, Jang, Kim, & Park, 2018 and for a detailed look at the number of papers related to AI techniques and their clinical applications see (Tran et al., 2019) 3 Figure 1. AI Knowledge Map (AIKM). Source: Corea (2019), reproduced with permission courtesy of F. Corea. If harnessed effectively, such AI-clinician coordination (AI-Health) could offer great opportunities for the improvement of healthcare services and ultimately patients' health (Taddeo & Floridi, 2018) by significantly improving human clinical capabilities in diagnosis (Arieno, Chan, & Destounis, 2019; De Fauw et al., 2018; Kunapuli et al., 2018), drug discovery (Álvarez-Machancoses & Fernández-Martínez, 2019; Fleming, 2018), epidemiology (Hay, George, Moyes, & Brownstein, 2013), personalised medicine (Barton et al., 2019; Cowie, Calveley, Bowers, & Bowers, 2018; Dudley, Listgarten, Stegle, Brenner, & Parts, 2015) or operational efficiency (H. Lu & Wang, 2019; Nelson, Herron, Rees, & Nachev, 2019). However, as Ngiam & Khor (2019) stress, if these AI solutions are to be embedded in clinical practice, then at least three issues need to be considered: the technical possibilities and limitations; the ethical, regulatory and legal framework; and the governance framework. The task of the following pages is to focus on the second of these elements - the ethical, regulatory and legal framework - by stressing how important it is that the ethical challenges raised by 4 implementing AI in healthcare settings are tackled proactively (Char, Shah, & Magnus, 2018). If they are not, there is a risk of incurring significant opportunity costs (Cookson, 2018) due to what Floridi terms a 'double bottleneck' whereby "a double bottleneck: ethical mistakes or misunderstandings may lead to social rejection and/or distorted legislation and policies, which in turn may cripple the acceptance and advancement of [the necessary] data science".2 Although essential, encouraging this kind of proactive ethical analysis is challenging because – although bioethical principles for clinical research and healthcare are well established, and issues related to privacy, effectiveness, accessibility and utility are clear (Nebeker, Torous, & Bartlett Ellis, 2019) – other issues are less obvious (Char et al., 2018). For example, AI processes may lack transparency, making accountability problematic, or may be biased, and leading to unfair behaviour or mistaken decisions (Mittelstadt, Allo, Taddeo, Wachter, & Floridi, 2016). Identification of these less obvious concerns requires input from the medical sciences, economics, computer sciences, social sciences, law, and policy-making. Yet, research in these areas is currently happening in siloes, is overly focused on individual level impacts (Morley & Floridi, 2019b), or does not consider the fact that the ethical concerns may vary depending on the stage of the algorithm development pipeline (Morley, Floridi, Kinsey, & Elhalal, 2019). Taken together, these issues are inhibiting the development of a coherent ethical framework. Whilst AI-Health remains in the early stages of development and relatively far away from having a major impact on frontline clinical care (Panch, Mattie, & Celi, 2019) there is still time to develop this framework. However, this window of opportunity is closing fast, as the pace at which AIHealth solutions are gaining approval for use in clinical care in the US is accelerating (Topol, 2019). Both the Chinese (Zhang, Wang, Li, Zhao, & Zhan, 2018) and British governments (Department of Health and Social Care, 2019) have made it very clear that they intend on investing heavily in the spread and adoption of AI-Health technologies. It is for these reasons that the goal of this article is to offer a cross-disciplinary systematic review mapping the potential ethical implications of the development of AI-Health in order to support the development of better design practices, and transparent and accountable deployment strategies. We will do this in terms of digital ethics. That is, we will focus on the evaluation of moral problems related to data, algorithms and corresponding practices (Floridi & Taddeo, 2016), with the hope of enabling governments and healthcare systems looking to adopt AIHealth to be ethically mindful (Floridi, 2019a). 2 https://digitalethicslab.oii.ox.ac.uk/the-ethics-of-medical-data-analytics-opportunities-and-challenges/ 5 2. Methodology in Brief A detailed outline of the methodology used to conduct the review can be found in the appendix. For now, suffice to say that a traditional thematic review methodology (following Abdul, Vermeulen, Wang, Lim, & Kankanhalli, 2018) was used to find literature from across disciplinary boundaries that highlighted ethical issues unique to the use of AI algorithms in healthcare. This means that the review did not focus on issues such as lack of evidence, privacy and security (Vayena, Tobias, Afua, & Allesandro, 2018), or definitions and secondary uses of healthcare data, as these are ethical issues for digital health at large and not unique to AI. More detailed discussion of these issues is highlighted in Table 1. General Digital Health Ethical Concern Example References Data Sharing/Data Privacy (Kalkman, Mostert, Gerlinger, Van Delden, & Van Thiel, 2019)(Olimid, Rogozea, & Olimid, 2018) (Parker, Halter, Karliychuk, & Grundy, 2019)(Quinn, 2017) (Richardson, Milam, & Chrysler, 2015) (Townend, 2018) Secondary use of Healthcare Data (Lee, 2017) (Nittas, Mütsch, Ehrler, & Puhan, 2018) (O'Doherty et al., 2016) Surveillance, Nudging and Paternalism (Maher et al., 2019) (Marill, 2019)(Nebeker et al., 2017) (Morley & Floridi, 2019d)(Burr, Mariarosaria, & Floridi, 2019) Consent (S. Millett & O'Leary, 2015) (T. Ploug & Holm, 2016) (Mann, Savulescu, & Sahakian, 2016) (Balthazar, Harri, Prater, & Safdar, 2018) Definition of Health Data (Floridi et al., 2018) (Voigt, 2019) (Holzinger, Haibe-Kains, & Jurisica, 2019) (Kleinpeter, 2017) Ownership of Health Data (Chiauzzi & Wicks, 2019) (Krutzinna, Taddeo, & Floridi, 2018) (Shaw, Gross, & Erren, 2016) (Sterckx, Rakic, Cockbain, & Borry, 2016) (Stephan Millett & O'Leary, 2015) Embodied Intelligence/Robotics (Fiske, Henningsen, & Buyx, 2019) Digital Divide/eHealth Literacy (Celi et al., 2016) (Kuek & Hakkennes, 2019) Patient involvement (Aitken et al., 2019) (Page, Manhas, & Muruve, 2016) Patient Safety (Barras, 2019) Evidence of Efficacy (Ferretti, Ronchi, & Vayena, 2019) (Henson, David, Albright, & Torous, 2019) (Larsen et al., 2019) Table 1: Example literature related to ethical concerns that are relevant for all digital health intervention, not unique to AI-Health and therefore not included in this review To ensure that the focus stayed on the unique ethical issues, the map, developed by (Mittelstadt et al., 2016), of the epistemic, normative, and overarching ethical concerns related to algorithms was used as a base. First, the selected literature was reviewed to identify healthcare examples of each of the 6 concerns highlighted in the original map, as shown in Table 2, and then reviewed more thoroughly to identify how the ethical issues may vary depending on whether the analysis was being conducted at: (i) individual, (ii) interpersonal, (iii) group (e.g. family or population), (iv) institutional, (v) sectoral, and/or (v) societal levels of abstraction (LoA)3 (Floridi, 2008). This helped the review avoid the narrow focus on individual-level impacts highlighted in the introduction. This approach is not intended to imply that there is no overlap between the levels. Ethical Concern Explanation Medical Example Epistemic concerns Inconclusive Evidence Algorithmic outcomes (e.g. classification) are probabilistic and not infallible. They are rarely sufficient to posit the existence of a causal relationship. EKG readers in smartwatches may 'diagnose' a patient as suffering from arrhythmia when it may be due to a fault with the watch not being able to accurately read that user's heartbeat (for example due to the colour of their skin) or the 'norm' is inappropriately calibrated for that individual (Hailu, 2019) Inscrutable Evidence Recipients of an algorithmic decision very rarely have full oversight of the data used to train or test an algorithm or the data points used to reach a specific decision. A clinical decision support system deployed in a hospital may make a treatment recommendation, but it may not be clear on what basis it has made that 'decision' raising the risk that it has used data that are inappropriate for the individual in question or that there is a bug in the system leading to issues with over or under prescribing (Wachter, 2015). Misguided Evidence Algorithmic outcomes can only be as reliable (but also as neutral) as the data they are based on. Watson for Oncology is in widespread use in China for 'diagnosis' via image recognition but has primarily been trained on a Western data set leading to issues with concordance and poorer results for Chinese patients than their Western counterparts (Liu et al., 2018). Normative Concerns Unfair outcomes An action can be found to having more of an impact (positive or negative) on one group of people An algorithm 'learns' to prioritise patients it predicts to have better outcomes for a particular disease. This turns out to have a discriminatory effect on people within the Black and minority ethnic communities (Garattini, Raffle, Aisyah, Sartain, & Kozlakidis, 2019). Transformative effects Algorithmic activities, like profiling, re-conceptualise reality in unexpected ways. An individual using personal health app has limited oversight over what passive data it is collecting and how that is being transformed into a recommendation to improve, limiting their ability to challenge any recommendations made and a loss of personal autonomy and data privacy (Kleinpeter, 2017). Overarching Traceability Harm caused by algorithmic activity is hard to debug (to detect the harm and find its cause), and it is hard to identify who should be held responsible for the harm caused. If a decision made by clinical decision support software leads to a negative outcome for the individual, it is unclear who to assign the responsibility and or liability to and therefore to prevent it from happening again (Racine, Boehlen, & Sample, 2019).. Table 2: A summary of the epistemic, normative and overarching ethical concerns related to algorithmic use in healthcare based on Mittelstadt et al (2016) from (Jessica Morley & Floridi, In Press) . 3 A level of abstraction can be imagined as an interface that enables one to observe some aspects of a system analysed, while making other aspects opaque or indeed invisible. For example, one may analyse a house at the LoA of a buyer, of an architect, of a city planner, of a plumber, and so on. LoAs are common in computer science, where systems are described at different LoAs (computational, hardware, user-centred etc.). Note that LoAs can be combined in more complex sets, and can be, but are not necessarily hierarchical, with higher or lower 'resolution' or granularity of information. 7 3. Thematic Analysis What follows is a detailed discussion of the issues uncovered. A summary table (Table 3) is provided at the end of the section. 3.1. Epistemic Concerns: Inconclusive, Inscrutable, and Misguided Evidence Many factors are encouraging the development of AI-Health (Chin-Yee & Upshur, 2019). One of the main driving forces is the belief that algorithms can make more objective, robust and evidence-based clinical decisions (in terms of diagnosis, prognosis or treatment recommendations) than a human health care provider (HCP) can (Kalmady et al., 2019). This is not an unfounded position. Machine learning methods, especially ensemble and unsupervised methods (Harerimana et al., 2018), can take into account a far greater range of evidence (data) than a Health Care Provider (HCP) when making a clinical decision, including five of the seven dimensions of healthcare data provided by the US Department of Health and Human services: (1) demographic and socioeconomic data; (2) symptom and existing diagnosis data; (3) treatment data; (4) outcome data; and (5) other omic data (Holzinger et al., 2019)4. If designed taking into account the multiple epistemic concerns, this ability enables clinical algorithms to act as digital companions (Morley & Floridi, 2019d), reducing the information asymmetry that exists between a HCP and the individual seeking care by making available information accessible to both parties and helping ensure that the most informed decision possible is made by the person who has the right to make it (Morley & Floridi, 2019a). It is at least in part due to this ability to make 'evidence-based' decisions that, as AI-health research has shown, AI techniques can considerably augment or surpass human capabilities when it comes to tasks including: (1) analysis of risk factors (De Langavant, Bayen, & Yaffe, 2018; Deng, Luo, & Wang, 2018); (2) prediction of disease (Moscoso et al., 2019); (3) prediction of infection (Barton et al., 2019)(López-Martínez, Núñez-Valdez, Lorduy Gomez, & García-Díaz, 2019); (4) population health monitoring (F. S. Lu, Hattab, Clemente, Biggerstaff, & Santillana, 2019; Zacher & Czogiel, 2019); (5) prediction of adverse effects (Ding, Tang, & Guo, 2019; Mortazavi et al., 2017); (6) prediction of outcome and/or likelihood of survival (Dong et al., 2019; Popkes et al., 2019; Topuz, Zengul, Dag, Almehmi, & Yildirim, 2018); and (7) analysing electronic health records (Shickel, Tighe, Bihorac, & Rashidi, 2018). These capabilities should not be underestimated, particularly as AI-Health solutions can operate at scale, diagnosing or predicting outcomes for multiple people at once – 4 The other two categories refer to data from the Healthcare system, such as expenditure and healtchcare resources data. 8 something that an HCP could never do. Yet in many ways this almost unwavering faith in the truthtelling power of AI-Health is flawed. As has been highlighted multiple times in the wider ethical AI literature, the belief that algorithms are more objective than humans is a 'carefully crafted myth' (Gillespie, Boczkowski, & Foot, 2014), and just because an algorithm can recognise a pattern (for example) does not necessarily make it meaningful (Floridi, 2014). In the context of healthcare, existing methods and studies (potentially including those referenced) suffer from overfitting due to small numbers of samples, meaning that the majority of results (e.g. patterns of disease risk factors, or presence of disease) are inconclusive (Holzinger et al., 2019). This is a problem that is further magnified by the lack of reproducibility, and external validity, of results. AI-Health solutions are often untranslatable between different settings and rarely work in settings different to those in which the initial result was obtained (Vollmer, Mateen, Bohner, Király, Ghani, et al., 2018), raising serious questions about the scientific rigor of AI-Health and its safety (Vayena, Blasimme, & Cohen, 2018). Furthermore, the results can often be heavily value-laden, based on the definition of 'healthy' by influential people or powerful companies (McLaughlin, 2016). This raises a number of significant ethical concerns. At the individual LoA there is considerable risk of misdiagnosis. This can happen in at least two ways: either, as highlighted in table 2, by an individual using a wearable device that has a bug, or is inappropriately calibrated for them could be 'told' that they are suffering from a health condition when they are not (or vice versa) or, an HCP relying on clinical decision support software (CDSS) (Ruckenstein & Schüll, 2017) could be given an inaccurate diagnosis or recommendation which they do not question due to a tendency to uncritically accept the decisions of automated systems (Challen et al., 2019). Moreover, this can have impacts in medical practice, causing overreliance on the machine diagnostics and deskilling of practitioners (Cabitza, Rasoini, & Gensini, 2017). Not only is this a risk for individuals, but it also reverses the advantage of AI-Health solutions being able to operate at scale by introducing the group LoA ethical concern of misdiagnosis or missed diagnosis happening repeatedly. Whilst an HCP might give one person the wrong diagnosis and then be corrected, a faulty algorithm, based on the misguided, inscrutable or inconclusive evidence could give the same wrong diagnosis to hundreds or thousands of people at a time (Topol, 2019). The scale of the problems is as large as the scale of the solutions. Building on this, there are also ethical implications at the interpersonal LoA. HCP-patient relationships are primarily based on trust and empathy, and whilst AI-Health solutions can take over tasks that are more routine and standardised, they cannot reproduce the emotional virtues of which human HCPs are capable (Ngiam & Khor, 2019). Consequently, an over-reliance on the 'quantitative' 9 and objective evidence that fuels clinical algorithms (Cabitza et al., 2017) could discredit other forms of diagnosis (Rosenfeld et al., 2019) – a prominent concern in the case of clinical psychiatry (Burns, 2015) – and lead to the de-humanisation or impersonalisation of care provision (Juengst, McGowan, Fishman, & Settersten, 2016), from a service based on listening and theory to one based purely on categorisation (an issue that could again lead to a group LoA harm of group-profiling and associated discrimination by providers including insurers, see section 3.2.). Not only is this effectively 'paternalism in disguise' (Juengst et al., 2016) but it could also lead to poorer health outcomes due to the lack of disconnect between pure medical evidence and actual behaviour change (Emanuel & Wachter, 2019). Finally, scaling up to the institutional, sectoral and societal LoAs, there is the concern that public health decisions are increasingly made on predictive AI-Health algorithms, which too often rely on the same flawed assumptions as outlined above. Regarding these assumptions, consider what is by now a classic example: the Google Flu Trends monitoring of the influenza virus. The initial algorithm distorted the spread of the virus in the US (Vayena, Salathé, Madoff, & Brownstein, 2015). If policy decisions about where to deploy health resources are based on such poor-quality evidence, this could result in the waste of public funds (e.g., promoting vaccination campaigns where they are not needed), damage local economies (e.g., scaring away tourists from a region) – which would result in a positive feedback loop of less money available for public expenditure – and lead to poorer quality public healthcare provision and thus worse health outcomes for society at large. This worry is particularly paramount when it is considered that the ultimate ambition of AI-Health is to create a learning healthcare system where the 'system' is constantly learning from the data it receives on the performance of its interventions (Faden et al., 2013). Furthermore, it is worth noting that, at this juncture, the example offered above of Flu Trends does not represent the limits of Google's interest – and that of its subsidiaries and its siblings under parent company Alphabet – in public health. As we discuss below, the engagement between Alphabet's artificial intelligence subsidiary DeepMind and a major UK hospital has attracted the attention of data protection regulators, the press, and academics (Information Commissioner, 2018; Powles & Hodson, 2017). The challenge of ensuring that AIHealth systems function accurately has in turn sparked debates about the appropriateness of sharing data between public and private entities. In response to claims that patient data transferred from the Royal Free Hospital to DeepMind was "far in excess of the requirements of those publicly stated needs" (Powles & Hodson, 2017), DeepMind representatives argued that "data processed in the application have been defined by and are currently being used by clinicians for the direct monitoring 10 and care of AKI [acute kidney injury] patients" (King et al., 2018). Powles and Hodson responded in turn that it is a "statement of fact that the data transferred is broader than the requirements of AKI" (Powles & Hodson, 2018). As this series of claims and counter-claims demonstrates, the quality and quantity of data required for a particular AI-Health application is likely to be a matter of dispute in the context of the collection and sharing of patient data in training AI-Health. Ultimately, data is necessary for medical practice and thus so are AI-Health solutions that can take in greater volumes of data. But data collected and used in this way is insufficient to inform medical practice; it must be transformed to be useful (Car, Sheikh, Wicks, & Williams, 2019) and if this transformation process is flawed the results could be hugely damaging, resulting in either wasted funds and poorer health provision, or undue sharing of patient data with private sector actors under the guise of AI-Health. 3.2. Normative Concerns: Unfair Outcomes and Transformative Effects As referenced in the introduction, healthcare systems across the globe are struggling with increasing costs and decreasing outcomes (Topol, 2019) and their administrators increasingly believe that the answer may well lie in making healthcare systems more informationally mature and able to capitalise on the opportunities presented by AI-Health significantly to improve outcomes for patients, and to reduce the burdens on the system (Cath, Wachter, Mittelstadt, Taddeo, & Floridi, 2017). Whilst it would be ethically remiss to ignore these opportunities (Floridi, 2019a), it would be equally ethically problematic to ignore the fact that these opportunities are not created by AI-Health technologies per se but by their ability to re-ontologise (that is, fundamentally transform the intrinsic nature of) the ways in which health care is delivered by coupling, re-coupling and de-coupling different parts of the system (Floridi, 2017a). For example (Morley & Floridi, 2019b): • Coupling: patients and their data are so strictly and interchangeably linked that the patients are their genetic profiles, latest blood results, personal information, allergies etc. (Floridi, 2017a). What the legislation calls 'data subjects" become "data patients"; • Re-Coupling: research and practice have been sharply divided since the publication of the National Commission for the Protection of Human Subjects in the 1970s, but in the digital scenario described above, they are re-joined as one and the same again (Petrini, 2015) (Faden et al., 2013); • De-Coupling: presence of Health Care Provider (HCP) and location of Patient become independent, for example because of the introduction of online consultations (NHS England, 2019). 11 As a result of these transformations a number of ethical concerns arise. Starting once again with the individual LoA: as more diagnostic and therapeutic interventions become based on AI-Health solutions, individuals may be encouraged to share more and more personal data about themselves (Racine et al., 2019) - data that can then be used in opaque ways (Sterckx et al., 2016). This means that the ability for individuals to be meaningfully involved in shared decision making is considerably undermined (Vayena et al., 2018) As a result, the increasing use of algorithmic decision-making in clinical settings can have negative implications for individual autonomy, as for an individual to be able to exert agency over the AI-Health derived clinical decision, they would need to have a good understanding of the underlying data, processes and technical possibilities that were involved in it being reached (DuFault & Schouten, 2018) and be able to ensure their own values are taken into consideration (McDougall, 2019). The vast majority of the population do not have the level of eHealth literacy necessary for this (Kim & Xie, 2017), and those that do (including HCPs) are prevented from gaining this understanding due to the black-box nature of AIHealth algorithms (Watson et al., 2019). In extreme instances, this could undermine an individual's confidence in their ability to refuse treatment (Thomas Ploug & Holm, 2019). Such issues pose a substantial threat to an individual's integrity of self (the ability of an individual to understand the forces acting on them) (Cheney-Lippold, 2017). Given that damage to a person's psychological integrity can be perceived as a 'harm', not accounting for this potentiality poses the risk of creating a system that violates the first principle of medical ethics: primum non nocere ("first, do no harm") (Andorno, 2004; Morley & Floridi, 2019d). It is not necessarily the case that harmful impacts will primarily be felt by the patients. At the interpersonal LoA, HCPs may themselves feel increasingly left 'out of the loop' as decisions are made by patients and their 'clinical advice' algorithm in a closed digital loop (Nag, Pandey, Oh, & Jain, 2017). As a result, HCPs may too feel unable to exert their own agency over the decision-making capacity of AI-Health solutions. Though the use of algorithmic decision-making makes diagnostics seem like a straightforward activity of identifying symptoms and fitting them into textbook categories, medical practice is much less clear-cut than it seems (Cabitza et al., 2017). Clinical practice involves a series of evaluations, trial and error, and a dynamic interaction with the patient and the medical literature. As a result, formal treatment protocols should be seen more as evaluative guidelines than well-defined, isolated categories. AI-Health solutions may not be in accordance with current best practice, which is 12 necessary to handle the great degree of uncertainty5 and can only be fully evaluated by physicians (Cabitza et al., 2017). Therefore, AI-Health solutions need to allow HCPs to exert influence in the decision-making process. At the group LoA the concern is that AI-Health systems may well be able to cope better with illnesses and injuries that have well-established and fairly set (and therefore automatable) treatment protocols. These are more likely to exist for afflictions most commonly suffered by white men as there is a greater volume of medical trials data for this group than there is for almost any other group. Algorithms trained on such biased datasets could make considerably poorer predictions for, for example, younger black women (Vayena et al., 2018). If HCPs are left out of the loop completely and learning healthcare systems primarily rely on automated decisions, there is considerable potential to exacerbate existing inequalities between the "haves" and the "have-nots" of the digital healthcare ecosystem, i.e., those that generate enough data on themselves to ensure accurately trained algorithms and those that do not (Topol, 2019). To mitigate these and associated risks, institutions need to be asking the crucial question: how much clinical decision-making should we be delegating to AI-Health solutions (Di Nucci, 2019)? If it is known that algorithms which enable profiling (e.g. those that determine genetic risk profiles) can ignore outliers and provide the basis for discrimination (Garattini et al., 2019), so deciding whether healthcare also ought to be a means to promote social justice is crucial in order to establish what type of data services will be embedded in the system (Voigt, 2019), what data should be collected, and which values should be embedded in algorithmic decision-making services (McDougall, 2019). This decision also determines what sort of population-level behavioural change the health system should be able to aim for depending on cost management, data collection and fairness in data-driven systems (Department of Health and Social Care, 2018.). If not carefully considered, this process of transforming the provision of care risks over-fitting the system to a specific set of values that may not represent those of society at large (McDougall, 2019). Another, more subtle yet pervasive transformative effect arises at the sectoral level. Powles & Hodson (2017) argue that one risk that may arise from collaboration between public and private sector entities such as that between the Royal Free London hospital and DeepMind is that the positive 5 Here we are discussing fairly routine illnesses and injuries that have set treatment protocols that may need to be flexibly interpreted on a case-by-case basis. We recognize that there are other instances, such as in the case of rare diseases, where algorithmic systems might be better equipped to deal with diagnostic uncertainty (for example in cases of rare disesaes) by being able to draw on a wider range of data points and information sources than a human clinician could. 13 benefits of AI-Health "solutions" will be siloed within private entities. They note that in the Royal Free case, "DeepMind [was given] a lead advantage in developing new algorithmic tools on otherwise privately-held, but publicly-generated datasets" (Powles & Hodson, 2017, p. 362). This, they suggest, may mean that the only feasible way that future advances may be developed is "via DeepMind on DeepMind's terms". This interpretation was contested by DeepMind, who called it "unevidenced and untrue" and claimed that the Information Commissioner agreed with their stance in her 2018 ruling (King et al., 2018). Whatever the circumstances of this particular case, the broader risk of privately held AI-Health solutions – trained on datasets that have been generated about the public by public actors but then (lawfully) shared with private companies – is worthy of caution going forward. As may now be clear, these transformative effects also have significant ethical implications at the societal LoA. Before institutions can establish where and how (and, from the sectoral perspective, whether) AI-Health solutions can improve care, society itself must make difficult decisions about what care is and what constitutes good care (Coeckelberg, 2014). To offer a simplistic example, does it mean purely providing a technical diagnosis and an appropriate prescription or does it involve contemplating a series of human necessities that revolve around well-being (Burr, Taddeo, & Floridi, 2019)? If it is the former, then it is relatively easy to automate the role of non-surgical clinicians through AI (although this does not imply that doctors should be substituted by AI systems). However, if is the latter, then providing good health care means encompassing psychological wellbeing and other elements related to quality of life, which would make human interaction an essential part of healthcare provision, as a machine does not have the capability to make emotionally-driven decisions. Consequently, certain decisions may completely exceed the machine's capabilities and thus delegating these tasks to AIHealth would be ethically concerning (Matthias, 2015). Consider, for example, a situation where an AI-Health solution decides which patients are sent to the Intensive Care Unit (ICU). Intensive care is a limited resource and only people who are at risk of losing their lives or suffering grave harms are sent there. Triage decisions are currently made by humans with the aim of maximising well-being for the greatest number of people. Doctors weigh different factors when making this decision, including the likelihood of people surviving if they are sent to the ICU. These situations often involve practitioners (implicitly) taking moral stances, by prioritising individuals based on their age or health conditions. These cases are fundamentally oriented by legal norms and medical deontology, yet personal expertise, experience and values also inevitably play a role. Having the support of AI-Health in the ICU screening increases the number of agents and complicates the norms involved in these decisions, since the doctor may follow his or her professional guidelines, while the algorithm will be oriented by the values embedded in its code. Unless there is a 14 transparent process for society to be involved in the weighing of values embedded in these decisionmaking tools (for instance, how is 'fair' provision of care defined?) (Cohen, Amarasingham, Shah, Xie, & Lo, 2014), then the use of algorithms in such scenarios could result in the overfitting of the health system to a specific set of values that are not representative of society at large. In response to this risk, some attempts have already been made to involve the public at large in decisions over the design and deployment of AI systems. In early 2019, the UK's Information Commissioner's Office and the National Institute for Health Research staged a series of "citizens' juries" to obtain the opinions of a representative cross-section of British society regarding the use of AI in health (Information Commissioner, 2019). The "juries" were presented with four scenarios, two relating to health - using AI to diagnose strokes, and using it to find potential matches for a kidney transplant - and another two relating to criminal justice. Notably, the juries "strongly favoured accuracy over explanation" in the two scenarios involving AI in health (National Institute for Health Research, 2019). This is just one example of research which attempted to obtain public opinion data regarding AI in health, and there are reasons to suppose that the apparent preference among participants for accurate over explainable AI systems reflects the high-stakes and fast-moving scenarios that were presented (as opposed to, say, the more routine illnesses and injuries we are focusing on here). Nonetheless, it demonstrates the plausibility and preferability of involving the public in designing AI-Health systems. To conclude this sub-section, the notion that AI-Health technologies are ethically neutral is unrealistic, and having them perform moral decision-making and enforcement may provoke immoral and unfair results (Rajkomar, Hardt, Howell, Corrado, & Chin, 2018). The direct involvement of the public in the design of AI-Health may help mitigate these risks. This should be borne in mind by all those involved in the AI-driven transformation of healthcare systems. 3.3 Overarching Concerns: Traceability The previous sub-section outlined how the increasing use of AI-Health is fundamentally transforming the delivery of healthcare and the ethical implications of this process, particularly in terms of potentially unfair outcomes. This transformation process means that healthcare systems now rely on a dynamic, cyclical and intertwined series of interactions between human, artificial and hybrid agents (Vollmer, Mateen, Bohner, Király, & Ghani, 2018)(Turilli & Floridi, 2009). This is making it increasingly challenging identify interaction-emerging risks and allocate liability, raising ethical concerns with regards to moral responsibility. 15 Moral responsibility involves both looking forward, where an individual, group or organisation is perceived as being in charge of guaranteeing a desired outcome, and looking backwards to appropriate blame and possibly redress, when a failure has occurred (Wardrope, 2015). In a wellfunctioning healthcare system, this responsibility is distributed evenly and transparently across all nodes so that the causal chain of a given outcome can be easily replicated in the case of a positive outcome, or prevented from repeating in the case of a negative outcome (Floridi, 2013, 2016). In an algorithmically-driven healthcare system, a single AI diagnostic tool might involve many people organising, collecting and brokering data, and performing analyses on it, making this transparent allocation of responsibility almost impossible. In essence, not only is the decision-making process of a single algorithm a black-box, but the entire chain of actors that participate in the end product of AIHealth solutions is extremely complex. This makes the entire AI-Health ecosystem inaccessible and opaque, making responsibility and accountability difficult. To unpack the ethical implications of this at-scale lack of traceability, let us take the example of a digital heart-rate monitor that 'intelligently' processes biological and environmental data to signal to its user their risk of developing a heart condition. At the individual LoA this process relies on what can be termed the 'digital medical gaze' (Morley & Floridi, 2019d) and is based on this micro-cycle of self-reflection adapted from (Garcia, Romero, Keyson, & Havinga, 2014): 1. Gaining Knowledge: Algorithm reads multi-omic dataset to determine risk of heart attack and providers individual with a 'heart health score' 2. Gaining Awareness: on the advice of the algorithm, individual starts monitoring their activity level and becomes aware of how active they are 3. Self-reflection: as directed by the algorithm the individual reflects on how much high fat food they are eating in a day and compares this to their optimal diet based on their genomic profile and their level of activity 4. Action: individual takes the advice of the algorithm and takes specific actions to improve their heart-health score e.g. starts regular exercise. If this does not work, and the individual still ends up experiencing heart failure, this process of algorithmic surveillance (Rich & Miah, 2014) risk creating an elaborate mechanism for victim-blaming (Danis & Solomon, 2013; McLaughlin, 2016). The individual may be seen as being a 'bad user' for failing to act upon the allegedly objective and evidence-based advice of the algorithm (see section 3.1), and may therefore be framed as being morally responsible for their poor health and not deserving of state-provided healthcare. Yet, due to the lack of traceability, there can be no certainty that the poor 16 outcome was due to the lack of action by the individual: it could be a faulty device, buggy code, or the result of biased datasets (Topol, 2019). Moreover, even if a negative outcome were to result purely from an individual disregarding the guidance, the adoption of digital infrastructure that enables failure to be ascribed to a morally 'culpable' individual is itself a matter of ethical concern. These new insights may enable lives to be saved and quality of life to be drastically improved, yet they also shift the ethical burden of 'living well' squarely onto newly accountable individuals. The ontological shift that this new infrastructure permits - from individuals-as-patients deserving quality healthcare, regardless of their prior choices as fallible humans, to individuals-as-agents expected to take active steps to pre-empt negative outcomes - raises stark questions for bioethics, which has traditionally been seen as an "ethics of the receiver" (Floridi 2008). Moreover, these technological changes might prompt a shift in the ethical framework, burdening the individuals, while not providing de facto means of behavioural change. Many concerns stem from socio-demographic issues which entail harmful habits, and cannot oversimplified to a matter of delivering the adequate information to the patient (Owens & Cribb, 2019). Due to issues of bias (discussed further in section 3.2), there is, further, a group LoA ethical risk that some groups may come to be seen as being more morally irresponsible about their healthcare than others. Heart-rate monitors, for example, are notoriously less accurate for those with darker skin (Hailu, 2019), meaning that they could give considerably less accurate advice to people of colour than to those with light skin. If this results in people of colour being less able to use AI-Health advice to improve their heart-health, then these groups of people may be seen as morally reprehensible when it comes to their health. Furthermore, the healthcare could then 'learn' to predict that people of colour have worse heart-health, potentially resulting in these groups of individuals being discriminated against by, for example, insurers (Martani, Shaw, & Elger, 2019). At the interpersonal, institutional and sectoral LoAs, this moral responsibility translates into liability. If for example, instead of the heart-health algorithm providing the advice back to the individual, it provides the data to the individual's HCP and the HCP provides advice that either fails to prevent an adverse event or directly causes an adverse event, this could be the basis of a medical malpractice suit (Price, 2018). In this scenario, it remains unclear where the liability will eventually sit (Ngiam & Khor, 2019). Current law implies that the HCP would be at fault, and therefore liable, for an adverse event as the algorithm in this scenario would be considered a diagnostic support tool – just like a blood test – with no decision making capacity, so it is the HCP's responsibility to act appropriately based on the information provided (Price, Gerke, & Cohen, 2019; Schönberger, 2019; Sullivan & Schweikart, 2019). However, the supply chain for any clinical algorithm is considerably 17 more complex and less transparent than that of a more traditional diagnostic tool meaning that many are questioning whether this is actually how the law will be interpreted in the future. For example, does the liability really sit with the HCP for not questioning the results of the algorithm, even if they were not able to evaluate the quality of the diagnostic against other sources of information, including their own personal knowledge of the patient due to the black-box nature of the algorithm itself? And what about the role of the hospital or care facility: does it have a responsibility to put in place a policy allowing HCPs to overrule algorithmic advice when this seems indicated? Similarly, what role do commissioners or retailers of the device that contains the algorithm play? Do they not carry some responsibility for not checking its accuracy, or do they assume that this responsibility sits with the regulator (for example MHRA in the UK, the FDA in the US or the CFDA in China) who should, therefore, carry the burden for not appropriately assessing the product before it was deployed in the market? What if the problem is further back in the chain, stemming from inaccurate coding or poorquality training data? There is a clear lack of distributed responsibility (Floridi, 2013, 2016)– a problem that is exacerbated by a lack of transparency – making it hard to hold individual parts of the chain accountable for poor outcomes which poses a significant ethical risk. In their overview of patient-safety issues with AI in healthcare He et al. (2019) state that those working in the field are trying to establish a systems-wide approach that does not attribute blame to individuals or individual companies, but conclude that where liability will ultimately rest remains to be seen. This is problematic because, as Hoffman et al. (2019) stress, uptake of algorithmic-decisionmaking tools by the clinical community is highly unlikely until this liability question is resolved (Vayena et al., 2018), which could result in the overarching ethical concern raised in the introduction – that of a significant missed opportunity. Many, including (Holzinger et al., 2019) believe that explainability is the answer to solving this problem and that, if HCPs can understand how a decision was reached, then reflecting on the output of an algorithm is no different from any other diagnostic tool. Indeed Schönberger (2019) argues that legally this is the case and that as long as it can be proven that the duty of care was met, then harm caused to a patient by an erroneous prediction of an AI-Health system would not yet constitute medical negligence but that it might in the near future constitute negligence to not rely on the algorithmic output – which brings us back to the issues outlined in section 3.1. Overall, this lack of clarity will continue to persist for some time (Schönberger, 2019), making it once again a social issue. Society will ultimately dictate what the socially acceptable and socially preferable (Floridi & Taddeo, 2016) answers are to these pressing questions. The ethical issue is whether all parts of society will have an equal say in this debate, as in the example of citizens' juries above, or whether it will be those individuals or groups with the loudest voices that get to set the rules. 18 As (Beer, 2017) attests, when thinking about the power of an algorithm, we need to think beyond the impact and consequences of the code, to the powerful ways in which notions and ideas about the algorithm circulate throughout the social world. A. Individual B. Interpersonal C. Group D. Institutional E. Sectoral F. Societal 1. Epistemic concern (inconclusive, inscrutable and misguided evidence) Misdiagnosis or missed diagnosis Loss of trust in HCP-Patient relationships, depersonalisation of care Misdiagnosis or missed diagnosis at scale – some groups more affected than others Waste of funds and resources not directed to areas of greater need Excessively broad data sharing between public and private entities Poorer public healthcare provision and worsening health outcomes for society 2. Normative (transformative effects and unfair outcomes) Surveillance & undermining of autonomy and integrity of self Deskilling of HCPs, overreliance on AI-tools, and undermining of consent practices and redefining roles in the healthcare system Profiling and discrimination against certain groups seen as being less healthy or higher risk Transformation of care pathways & imposing of specific values at scale – redefining 'good care' Siloing of new AI tool development within private sector Inequalities in outcomes 3. Overarching (traceability) 'Bad Users' could come to be blamed for their own illhealth See institutional Specific groups framed as being more morally irresponsible with regards to their health than others Lack of clarity over liability with regards to issues with safety and effectiveness could halt adoption or result in certain groups being blamed more often than others See institutional Society must decide through regulation preferable answers to the questions regarding liability and risk allocation in healthcare provision. However, all groups in society may not be given an equal say in this process Table 3: summary of the epistemic, normative and overarching ethical concerns associated with AI-Health at the five different LoAs as identified by the literature review 4.The Need for an Ethically-Mindful and Proportionate Approach The literature surveyed in this review clearly indicates the need for an agreed pro-ethical blueprint for AI-Health that considers the epistemic, normative and traceability ethical concerns at the five different LoAs. Protecting people from the harms of AI-Health goes beyond protecting data collection and applying a valid model. Normative frameworks need to contemplate the complexities of the human interactions where these technologies will be introduced and their emotional impacts (Luxton, 2014; Riek, 2016). Importantantly, an adequate normative framework should deal with the key question related to liability allocation in cases of medical error. Much of the risk of handling data and algorithms stems from professionals not adopting measures to protect privacy and support cybersecurity. The solution of risk management will come not only from drawing the boundaries of responsibility, but also 19 promoting capacitation, understanding and interfaces for handling AI tools. For one, promoting doctors' and patients' understanding and control over how AI-Health produces predictions or recommendations that are used in treatment plans, and access to and protection of patient data (Ngiam & Khor, 2019). Also, there needs to be control over how the interface and design of AI-Health products influences HCP-patient-artificial-agent interactions (Cohen et al., 2014). Finally, a certification for professionals seeking to use AI-Health tools is also necessary for the adequate implementation and use of AI (Kluge, Lacroix, & Ruotsalainen, 2018). To tackle these challenges, healthcare systems will need to update outdated governance mechanisms (policies, standards and regulations). These can be replaced with both hard and soft mechanisms, meaning what ought to be done and what may be done based on the existing moral obligations (Floridi, 2018), that balance the need to protect individuals from harm, whilst still supporting innovation that can deliver genuine system and patient benefit (Morley & Joshi, 2019). In short, healthcare systems should not be overly cautious about the adoption of AI-Health solutions, but should be mindful of the potential ethical impacts (Floridi, 2019a) so that proportionate governance models can be developed (Sethi & Laurie, 2013) which can, in turn, ensure that those responsible for ensuring that healthcare systems are held accountable for the delivery of high-quality equitable and safe care, can continue to be so. What these regulations, standards and policies should cover and how they should be developed remain open questions (Floridi, 2017b) which will likely be 'solved' multiple times over by different healthcare systems operating in different settings. However, in order to lend a more systematic approach to addressing these outstanding questions, enabling greater coherence and speed in addressing these challenges, in Table 4 below we have assembled a list of essential cross-cutting considerations that emerge from our literature review. The table indicates from which aspect of our systematic review (ethical concern x LoA, corresponding to a cell in Table 3) each consideration is drawn. 20 Consideration Key supporting literature Relevant aspects (ascending LoA6) The professional skills required of the healthcare workforce, including information governance skills (Kluge et al., 2018) Epistemic (A, B, C, F) Normative (B, C, D, E) Overarching (A, C) Which tasks should be delegated to AI-Health solutions (Di Nucci, 2019) Epistemic (A, B, C, D, F) Normative (B, C, D, F) Overarching (A, C, D) What evidence is needed to 'prove' clinical effectiveness of an AI-Health solution (Greaves et al., 2018) Epistemic (A, B, C, E, F) Normative (E) Overarching (A, C, D, F) Mechanisms for reporting or seeking redress for AI-Health associated harms (Schönberger, 2019) Epistemic (A, C, E, F) Normative (A, C, E, F) Overarching (A, C, D) Mechanisms for the inclusion of all relevant stakeholder views in the development of AIHealth systems (Aitken et al., 2019) Epistemic (C, E, F) Overarching (A, C, D, F) Explainability of specific AI-Health decisions (Watson et al., 2019) Epistemic (A, C) Normative (A, C, E) Overarching (A, D) Reliability, replicability and safety of AIHealth solutions (Challen et al., 2019) Epistemic (A, C, F) Normative (C, E, F) Overarching (A, C, D) Transparency over how algorithmic tools are integrated into the healthcare workflow, how it shapes decisions, and how it affects process optimization within medical services (Effy Vayena et al., 2015) Epistemic (A, B, C, D, E, F) Normative (A, B, D, F) Overarching (A, D, F) How traditional and non-traditional sources of health data can be incorporated into AIHealth decision making, how it can be appropriately protected and how it can be harmonised (e.g. Maher et al., 2019; Ploug & Holm, 2016; Richardson, Milam, & Chrysler, 2015; Townend, 2018) Epistemic (A, C, D, E, F) Normative (A, C, D, E, F) Overarching (A, C, D, E) How bioethical concepts (beneficence, nonmaleficence, autonomy and justice (Beauchamp & Childress, 2013) are challenged by AI-Health (Mittelstadt, 2019) Epistemic (B, F) Normative (A, C, D, F) Overarching (A, F) How concepts such as fairness, accountability and transparency can be maintained at scale (Morley & Floridi, 2019b) so that, for example, the output of algorithmic diagnostics does not result in economic benefits to specific drug producers or technology companies (Rosenfeld et al., 2019) Epistemic (C, D, E, F) Normative (D, E, F) Overarching (F) Table 4: the 11 key considerations for policymakers that arose from the literature review. Awareness of the need to consider these questions is increasing, and efforts are being made at both a national and international level to adapt existing regulations so that they remain fit for purpose (The 6 Denoted by an increasing Level of Analysis: Individual (A), Interpersonal (B), Group (C), Institutional (D), Sectoral (E) and Societal (F). 21 Lancet Digital Health, 2019). The American Food and Drug Administration (FDA) is now planning on regulating Software as a Medical Devices (SaMD) (Food and Drug Administration (FDA), 2019) and in both the EU and the UK Regulation 2017/745 on medical devices comes into effect in April 2020 and significantly increases the range of software and non-medical products that will need to be classed (and assessed) as medical devices. Additionally, the UK has published its Code of Conduct for data-driven health and care technologies, standards for evidence of clinical effectiveness for digital health technologies (Greaves et al., 2018) – a digital assessment questionnaire standards for apps – and is currently developing a 'regulation as a service' model to ensure that there are appropriate regulatory checks at all stages of the AI development cycle (Morley & Joshi, 2019). The World Health Organisation has a number of projects under way to develop guidance for member states (Aicardi et al., 2016) (World Health Organisation, 2019). In China, several norms7 provide specific and detailed instructions to ensure health data security and confidentiality (Wang, 2019) to ensure that health and medical big data sets can be used as a national resource to develop algorithms (Zhang et al., 2018) for the improvement of public health (Li, Li, Jiang, & Lan, 2019). These are all steps in the right direction, however, their development is progressing slowly (which is why the relevant literature is unlikely to reflect all current developments) and almost all focus solely on interventions positioning themselves as being health-related in the medical sense, not in the wider, wellbeing sense, e.g., healthy exercise, diet, sleeping habits). They will not necessarily mitigate risks are associated with the expanding wellness industry, which provides algorithmic tools that potentially enable people to bypass formal and well-regulated healthcare systems entirely by accessing technology directly, either by using a wearable device or consulting online databases (Burr et al., 2019). Similarly, although some technical solutions have been put forward for mitigating issues with data bias (Gebru et al., 2018; Holland, Hosny, Newman, Joseph, & Chmielinski, 2018) and data quality (Dai, Yoshigoe, & Parsley, 2018) and ensuring social inclusion in decision making (Balthazar et al., 2018; Friedman, Hendry, & Borning, 2017; Rahwan, 2018), these remain relatively untested. Unless a competitive advantage of taking such pro-ethical steps becomes clear without these approaches being made mandatory, it is unlikely that they will have a significant impact on the ethical impacts of AIHealth in the near future. As a result, there is still little control over the procedures followed and quality 7 Article 6 of the Regulations on the Management of Medical Records of Medical Institutions, Article 8 of the Management Regulations on Application of Electronic Medical Records, Article 6 of the Measures for the Management of Health Information, the Cybersecurity Law of the People's Republic of China, and the new Personal Information Security Specification. 22 control mechanisms (Cohen et al., 2014) involved in the development, deployment and use of AIHealth. As these comparatively easier to tackle problems do not yet have adequate solutions, it is unsurprising that the bigger issues regarding the protection of equality of care (Powell & Deetjen, 2019, fair distribution of benefits (Balthazar et al., 2018) (Kohli & Geis, 2018) and the protection and promotion of societal values (Mahomed, 2018) have barely even been considered. Given that healthcare systems in many ways act as the core of modern societies this is concerning. If mistakes are made too early in the adoption and implementation of AI in healthcare, the fall-out could be significant enough to undermine public trust, resulting in significant opportunity costs, and potentially encouraging individuals to seek their healthcare from outside of the formal systems where they may be presented with even greater risks. A coherent approach is needed and urgently, hopefully this systematic overview of the issues to be considered can help speed up its development. 5. Conclusion This thematic literature review has sought to map out the ethical issues around the incorporation of data-driven AI technologies into healthcare provision and public health systems. In order to make this overview more useful, the relevant topics have been organised into themes and five different levels of abstraction (LoAs) have been highlighted. The hope is that by encouraging a discussion of the ethical implications of AI-Health at individual, interpersonal, group, institutional and societal LoAs, policymakers and regulators will be able to segment a large and complex conversation into tractable debates around specific issues, stakeholders, and solutions. This is important, as Topol (2019) states 'there cannot be exceptionalism for AI in medicine,' especially not when there is potentially so much to gain (Miotto, Wang, Wang, Jiang, & Dudley, 2018). With this in mind, the review has covered a wide range of topics while also venturing into the specificity of certain fields. This approach has made it to develop a fuller and more nuanced understanding of the ethical concerns related to the introduction of AI into healthcare systems than has been previously seen in the literature. Inevitably, there are limitations to this approach, which are specified in the appendix, detailing our methodology and pointing towards opportunities for further research. In this article, we hope to have provided a sufficiently comprehensive, detailed, and systematic analysis of the current debates on ethical issues related to the introduction of AI into healthcare systems. The aim is to help policymakers and legislators develop evidence-based and proportionate 23 policy and regulatory interventions. In particular, we hope to encourage the development of a system of transparent and distributed responsibility, where all those involved in the clinical algorithm supply chain can be held proportionately and appropriately accountable for the safety of the patient at the end, not just the HCP. It is only by ensuring such a system is developed that policymakers and legislators can be confident that the inherent risks we have described are appropriately mitigated (as far as possible) and only once this is the case will the medical community at large feel willing and able to adopt AI technologies. 24 Appendix – Methodology The data collection for this research was divided into three stages as outlined in the below schematic. This process resulted in approximately 147 papers suitable for analysis and inclusion in the initial review. Subsequent relevant papers that met the criteria were added at a later date during the writing up of the results. This literature review also included accessory readings and case studies that were encountered during the research process. This includes bibliography obtained from the references of the papers analysed, and case studies identified in the readings (e.g. the Deep Mind case study). It is our belief that these exploratory readings enrich our systematic approach by developing on interesting findings and topics identified throughout our investigation. It is important to note that the selection of articles and policy documents was restricted to those written in English. This means that some ethical issues will have been overlooked (e.g. those in Spanish-speaking countries or in China). Second, academic literature, much like regulation, tends to struggle to keep pace with technological development. This literature review did not seek to identify 25 ethical issues associated with specific use cases of AI first-hand, for example, by reviewing recently published studies available on pre-print servers such as arXiv, but instead focused on providing an overview of the ethical issues already identified. As a result, there may well be ethical concerns that are associated with more emergent use cases of AI for healthcare that we have not identified as they have not yet been discussed in formal peer-reviewed publications. To overcome these limitations, further research could seek to expand the literature review by including a wider range of search queries, and by taking a case-study approach to analysing the ethical issues of specific practices and then aggregating these. This could be complemented by a comprehensive review of the policies, standards and regulations in development in different healthcare systems across the globe to assess the extent to which these are likely to be effective at mitigating these ethical concerns. Technology Ethics/Concerns Field Algorithm* Fairness Health* Artificial Intelligence / AI Moral* Machine Learning Ethics Governance Table 5: Showing terms refined from Mittelstadt et al (2016) and selected to focus the literature search on publications focusing specifically on the ethics of AI for health. It is important to note that the search parameters were not exactly the same in all databases. Adaptation was necessary since not all databases operated with the same syntax or accepted the same number of search queries. As a result, the arrangement of Boolean operators and a search parameter were adapted to ensure that all possible combinations were covered. Database Search Query Results Titles Selected Titles Downloaded SCOPUS ethic* AND algorithm* AND health* 596 39 19 (ethic* AND ( "Artificial Intelligence" OR ai ) AND health* ) 239 37 15 ( moral* AND ( "Artificial Intelligence" OR ai ) AND health* ) 46 2 0 ( fair* AND ( "Artificial Intelligence" OR ai ) AND health* ) 122 6 3 (moral* OR ethic*) AND "machine learning" AND health* 91 14 9 ( fair* ) AND "machine learning" AND health* ) 70 5 3 Web of Science ((fair* OR moral* OR ethic*) AND ("machine learning" OR "Artificial Intelligence" OR "AI" OR algorithm*) AND health*) 668 45 26 Philpapers '"machine learning" AND health* 3 1 1 Artificial Intelligence AND health* AND ethic* 1000+ - - algorithm* AND health* AND ethic* 5 0 0 ethics AND "artificial intelligence" AND health 3 2 2 AI or Artificial Intelligence or Fair AND ethic or moral or health AND health8 9 0 0 Google Scholar ethics algorithms health 15,400 18 18 ethics of machine learning in health 21,300 11 10 ETHICS & HEALTH and at least one of: algorithm OR machine learning OR artificial intelligence OR AI 716,000 2 1 26 ETHICS & HEALTH and at least one of: algorithm OR AI 105,000 2 2 MORAL & HEALTH and at least one of: algorithm OR AI 26,900 2 1 FAIR & HEALTH And at least one of: algorithm OR AI 38,000 0 0 PubMed ETHICS & ARTIFICIAL INTELLIGENCE OR MACHINE LEARNING 34, 193 37 37 Table 6: Showing the final results from all searches. It is important to note that multiple search queries were made to cover all the combinations and the numbers in the table thus represent the sum of results, titles evaluated and downloaded (not all found files were accessible for download). It is also important to note that only the first 500 most relevant results from Google Scholar were reviewed and anything written before 2014 was excluded to make the number of results more manageable. Funding Taddeo and Floridi's work was partially supported by Privacy and Trust Stream – Social lead of the PETRAS Internet of Things research hub – PETRAS is funded by the UK Engineering and Physical Sciences Research Council (EPSRC), grant agreement no. EP/N023013/1. Caio's, Taddeo's and Floridi's work was also partially supported by a Microsoft grant and a Google grant. 27 References Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018). Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems CHI '18, 1–18. https://doi.org/10.1145/3173574.3174156 Aicardi, C., Del Savio, L., Dove, E. S., Lucivero, F., Tempini, N., & Prainsack, B. (2016). Emerging ethical issues regarding digital health data. On the World Medical Association Draft Declaration on Ethical Considerations Regarding Health Databases and Biobanks. Croatian Medical Journal, 57(2), 207–213. https://doi.org/10.3325/cmj.2016.57.207 Aitken, M., Tully, M. P., Porteous, C., Denegri, S., Cunningham-Burley, S., Banner, N., ... Willison, D. J. (2019). Consensus Statement on Public Involvement and Engagement with Data-Intensive Health Research. International Journal of Population Data Science, 4(1). https://doi.org/10.23889/ijpds.v4i1.586 Álvarez-Machancoses, Ó., & Fernández-Martínez, J. L. (2019). Using artificial intelligence methods to speed up drug discovery. Expert Opinion on Drug Discovery, 14(8), 769–777. https://doi.org/10.1080/17460441.2019.1621284 Andorno, R. (2004). The right not to know: An autonomy based approach. Journal of Medical Ethics, 30(5), 435– 439. https://doi.org/10.1136/jme.2002.001578 Arieno, A., Chan, A., & Destounis, S. V. (2019). A review of the role of augmented intelligence in breast imaging: From automated breast density assessment to risk stratification. American Journal of Roentgenology, 212(2), 259–270. https://doi.org/10.2214/AJR.18.20391 Balthazar, P., Harri, P., Prater, A., & Safdar, N. M. (2018). Protecting Your Patients' Interests in the Era of Big Data, Artificial Intelligence, and Predictive Analytics. Journal of the American College of Radiology, 15(3), 580–586. https://doi.org/10.1016/j.jacr.2017.11.035 Barakat, N., Bradley, A. P., & Barakat, M. N. H. (2010). Intelligible Support Vector Machines for Diagnosis of Diabetes Mellitus. IEEE Transactions on Information Technology in Biomedicine, 14(4), 1114–1120. https://doi.org/10.1109/TITB.2009.2039485 Barras, C. (2019). Mental health apps lean on bots and unlicensed therapists. Nature Medicine. https://doi.org/10.1038/d41591-019-00009-6 Bartoletti, I. (2019). AI in healthcare: Ethical and privacy challenges. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11526 LNAI, 7–10. https://doi.org/10.1007/978-3-030-21642-9_2 Barton, C., Chettipally, U., Zhou, Y., Jiang, Z., Lynn-Palevsky, A., Le, S., ... Das, R. (2019). Evaluation of a machine learning algorithm for up to 48-hour advance prediction of sepsis using six vital signs. Computers in Biology and Medicine, 109, 79–84. https://doi.org/10.1016/j.compbiomed.2019.04.027 Beauchamp, T. L., & Childress, J. F. (2013). Principles of biomedical ethics (7th ed). New York: Oxford University Press. Beer, D. (2017). The social power of algorithms. Information, Communication & Society, 20(1), 1–13. https://doi.org/10.1080/1369118X.2016.1216147 Brown, M. P. S., Grundy, W. N., Lin, D., Cristianini, N., Sugnet, C. W., Furey, T. S., ... Haussler, D. (2000). Knowledge-based analysis of microarray gene expression data by using support vector machines. Proceedings of the National Academy of Sciences, 97(1), 262–267. https://doi.org/10.1073/pnas.97.1.262 Burns, T. (2015). Our necessary shadow: The nature and meaning of psychiatry. Place of publication not identified: Pegasus Books. Burr, C., Mariarosaria, T., & Floridi, L. (2019, February 1). The Ethics of Digital Well-Being: A Thematic Review. Retrieved 19 April 2019, from SSRN website: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3338441&download=yes Cabitza, F., Rasoini, R., & Gensini, G. F. (2017). Unintended Consequences of Machine Learning in Medicine. JAMA, 318(6), 517. https://doi.org/10.1001/jama.2017.7797 Car, J., Sheikh, A., Wicks, P., & Williams, M. S. (2019). Beyond the hype of big data and artificial intelligence: Building foundations for knowledge and wisdom. BMC Medicine, 17(1). https://doi.org/10.1186/s12916-019-1382-x 28 Cath, C., Wachter, S., Mittelstadt, B., Taddeo, M., & Floridi, L. (2017). Artificial Intelligence and the 'Good Society': The US, EU, and UK approach. Science and Engineering Ethics. https://doi.org/10.1007/s11948-017-9901-7 Celi, L. A., Davidzon, G., Johnson, A. E. W., Komorowski, M., Marshall, D. C., Nair, S. S., ... Stone, D. J. (2016). Bridging the health data divide. Journal of Medical Internet Research, 18(12). https://doi.org/10.2196/jmir.6400 Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019). Artificial intelligence, bias and clinical safety. BMJ Quality & Safety, 28(3), 231–237. https://doi.org/10.1136/bmjqs-2018-008370 Char, D. S., Shah, N. H., & Magnus, D. (2018). Implementing Machine Learning in Health Care-Addressing Ethical Challenges. The New England Journal of Medicine, 378(11), 981–983. https://doi.org/10.1056/NEJMp1714229 Cheney-Lippold, J. (2017). We are data: Algorithms and the making of our digital selves. New York: New York University Press. Chiauzzi, E., & Wicks, P. (2019). Digital Trespass: Ethical and Terms-of-Use Violations by Researchers Accessing Data From an Online Patient Community. Journal of Medical Internet Research, 21(2), e11985. https://doi.org/10.2196/11985 Chin-Yee, B., & Upshur, R. (2019). Three Problems with Big Data and Artificial Intelligence in Medicine. Perspectives in Biology and Medicine, 62(2), 237–256. https://doi.org/10.1353/pbm.2019.0012 Coeckelberg, M. (2014). Good Healthcare Is in the "How": The Quality of Care, the Role of Machines, and the Need for New Skills. In Machine medical ethics (Vol. 74). New York: Springer. Cohen, I. G., Amarasingham, R., Shah, A., Xie, B., & Lo, B. (2014). The Legal And Ethical Concerns That Arise From Using Complex Predictive Analytics In Health Care. Health Affairs, 33(7), 1139–1147. https://doi.org/10.1377/hlthaff.2014.0048 Cookson, C. (2018, September 6). Artificial intelligence faces public backlash, warns scientist. Financial Times. Retrieved from https://www.ft.com/content/0b301152-b0f8-11e8-99ca-68cf89602132 Cowie, J., Calveley, E., Bowers, G., & Bowers, J. (2018). Evaluation of a digital consultation and self-care advice tool in primary care: A multi-methods study. International Journal of Environmental Research and Public Health, 15(5). https://doi.org/10.3390/ijerph15050896 Dai, W., Yoshigoe, K., & Parsley, W. (2018). Improving Data Quality through Deep Learning and Statistical Models. ArXiv:1810.07132 [Cs], 558, 515–522. https://doi.org/10.1007/978-3-319-54978-1_66 Danis, M., & Solomon, M. (2013). Providers, Payers, The Community, And Patients Are All Obliged To Get Patient Activation And Engagement Ethically Right. Health Affairs, 32(2), 401–407. https://doi.org/10.1377/hlthaff.2012.1081 De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., ... Ronneberger, O. (2018). Clinically applicable deep learning for diagnosis and referral in retinal disease. Nature Medicine, 24(9), 1342–1350. https://doi.org/10.1038/s41591-018-0107-6 De Langavant, L. C., Bayen, E., & Yaffe, K. (2018). Unsupervised machine learning to identify high likelihood of dementia in population-based surveys: Development and validation study. Journal of Medical Internet Research, 20(7). https://doi.org/10.2196/10493 Deng, X., Luo, Y., & Wang, C. (2018). Analysis of Risk Factors for Cervical Cancer Based on Machine Learning Methods. 2018 5th IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS), 631–635. https://doi.org/10.1109/CCIS.2018.8691126 Department of Health and Social Care. (n.d.-a). Annual Report of the Chief Medical Office 2018: Health 2040-Better Health Within Reach. Retrieved from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file /767549/Annual_report_of_the_Chief_Medical_Officer_2018_-_health_2040__better_health_within_reach.pdf Department of Health and Social Care. (n.d.-b). Health Secretary announces £250 million investment in artificial intelligence [Gov.uk]. Retrieved 8 August 2019, from https://www.gov.uk/government/news/health-secretary-announces-250-million-investment-inartificial-intelligence Di Nucci, E. (2019). Should we be afraid of medical AI? Journal of Medical Ethics. https://doi.org/10.1136/medethics-2018-105281 29 Ding, Y., Tang, J., & Guo, F. (2019). Identification of drug-side effect association via multiple information integration with centered kernel alignment. Neurocomputing, 325, 211–224. https://doi.org/10.1016/j.neucom.2018.10.028 Dong, R., Yang, X., Zhang, X., Gao, P., Ke, A., Sun, H., ... Shi, G. (2019). Predicting overall survival of patients with hepatocellular carcinoma using a three‐category method based on DNA methylation and machine learning. Journal of Cellular and Molecular Medicine, 23(5), 3369–3374. https://doi.org/10.1111/jcmm.14231 Dudley, J. T., Listgarten, J., Stegle, O., Brenner, S. E., & Parts, L. (2015). Personalized medicine: From genotypes, molecular phenotypes and the quantified self, towards improved medicine. 342–346. DuFault, B. L., & Schouten, J. W. (2018). Self-quantification and the datapreneurial consumer identity. Consumption Markets & Culture, 1–27. https://doi.org/10.1080/10253866.2018.1519489 Emanuel, E. J., & Wachter, R. M. (2019). Artificial Intelligence in Health Care: Will the Value Match the Hype? JAMA, 321(23), 2281–2282. https://doi.org/10.1001/jama.2019.4914 Faden, R. R., Kass, N. E., Goodman, S. N., Pronovost, P., Tunis, S., & Beauchamp, T. L. (2013). An Ethics Framework for a Learning Health Care System: A Departure from Traditional Research Ethics and Clinical Ethics. Hastings Center Report, 43(s1), S16–S27. https://doi.org/10.1002/hast.134 Ferretti, A., Ronchi, E., & Vayena, E. (2019). From principles to practice: Benchmarking government guidance on health apps. The Lancet Digital Health, 1(2), e55–e57. https://doi.org/10.1016/S25897500(19)30027-5 Fiske, A., Henningsen, P., & Buyx, A. (2019). Your Robot Therapist Will See You Now: Ethical Implications of Embodied Artificial Intelligence in Psychiatry, Psychology, and Psychotherapy. Journal of Medical Internet Research, 21(5), e13216. https://doi.org/10.2196/13216 Fleming, N. (2018). How artificial intelligence is changing drug discovery. Nature, 557(7707), S55–S57. https://doi.org/10.1038/d41586-018-05267-x Floridi, L., Luetge, C., Pagallo, U., Schafer, B., Valcke, P., Vayena, E., ... Kalra, D. (2018). Key Ethical Challenges in the European Medical Information Framework. Minds and Machines, 1–17. https://doi.org/10.1007/s11023-018-9467-4 Floridi, Luciano. (2008). The Method of Levels of Abstraction. Minds and Machines, 18(3), 303–329. https://doi.org/10.1007/s11023-008-9113-7 Floridi, Luciano. (2013). Distributed Morality in an Information Society. Science and Engineering Ethics, 19(3), 727–743. https://doi.org/10.1007/s11948-012-9413-4 Floridi, Luciano. (2014). The 4th revolution: How the infosphere is reshaping human reality. Oxford: Oxford Univ. Press. Floridi, Luciano. (2016). Faultless responsibility: On the nature and allocation of moral responsibility for distributed moral actions. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2083), 20160112. https://doi.org/10.1098/rsta.2016.0112 Floridi, Luciano. (2017a). Digital's Cleaving Power and Its Consequences. Philosophy & Technology, 30(2), 123– 129. https://doi.org/10.1007/s13347-017-0259-1 Floridi, Luciano. (2017b). The Logic of Design as a Conceptual Logic of Information. Minds and Machines, 27(3), 495–519. https://doi.org/10.1007/s11023-017-9438-1 Floridi, Luciano. (2018). Soft ethics, the governance of the digital and the General Data Protection Regulation. Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences, 376(2133). https://doi.org/10.1098/rsta.2018.0081 Floridi, Luciano. (2019a). AI opportunities for healthcare must not be wasted. Health Management, 19. Floridi, Luciano. (2019b). What the Near Future of Artificial Intelligence Could Be. Philosophy & Technology, 32(1), 1–15. https://doi.org/10.1007/s13347-019-00345-y Floridi, Luciano, & Taddeo, M. (2016). What is data ethics? Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2083), 20160360. https://doi.org/10.1098/rsta.2016.0360 Friedman, B., Hendry, D. G., & Borning, A. (2017). A Survey of Value Sensitive Design Methods. Foundations and Trends® in Human–Computer Interaction, 11(2), 63–125. https://doi.org/10.1561/1100000015 Garattini, C., Raffle, J., Aisyah, D. N., Sartain, F., & Kozlakidis, Z. (2019). Big Data Analytics, Infectious Diseases and Associated Ethical Impacts. Philosophy & Technology, 32(1), 69–85. https://doi.org/10.1007/s13347-017-0278-y 30 Garcia, J., Romero, N., Keyson, D., & Havinga, P. (2014). Reflective healthcare systems: Mirco-cylce of selfreflection to empower users. Interaction Design and Architecture(s), 23(1), 173–190. Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumeé III, H., & Crawford, K. (2018). Datasheets for Datasets. ArXiv:1803.09010 [Cs]. Retrieved from http://arxiv.org/abs/1803.09010 Gillespie, T., Boczkowski, P. J., & Foot, K. A. (2014). Media technologies: Essays on communication, materiality, and society. Cambridge, Massachusetts: The MIT Press. Greaves, F., Joshi, I., Campbell, M., Roberts, S., Patel, N., & Powell, J. (2018). What is an appropriate level of evidence for a digital health intervention? The Lancet, 392(10165), 2665–2667. https://doi.org/10.1016/S0140-6736(18)33129-5 Hailu, R. (2019). Fitbits and other wearables may not accurately track heart rates in people of color. STAT. Retrieved from https://www.statnews.com/2019/07/24/fitbit-accuracy-dark-skin/ Harerimana, G., Jang, B., Kim, J. W., & Park, H. K. (2018). Health Big Data Analytics: A Technology Survey. IEEE Access, 6, 65661–65678. https://doi.org/10.1109/ACCESS.2018.2878254 Hay, S. I., George, D. B., Moyes, C. L., & Brownstein, J. S. (2013). Big data opportunities for global infectious disease surveillance. He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019). The practical implementation of artificial intelligence technologies in medicine. Nature Medicine, 25(1), 30–36. https://doi.org/10.1038/s41591018-0307-0 Henson, P., David, G., Albright, K., & Torous, J. (2019). Deriving a practical framework for the evaluation of health apps. The Lancet Digital Health, 1(2), e52–e54. https://doi.org/10.1016/S2589-7500(19)30013-5 Hoffman, L., Benedetto, E., Huang, H., Grossman, E., Kaluma, D., Mann, Z., & Torous, J. (2019). Augmenting Mental Health in Primary Care: A 1-Year Study of Deploying Smartphone Apps in a Multi-site Primary Care/Behavioral Health Integration Program. Frontiers in Psychiatry, 10, 94. https://doi.org/10.3389/fpsyt.2019.00094 Holland, S., Hosny, A., Newman, S., Joseph, J., & Chmielinski, K. (2018). The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards. ArXiv:1805.03677 [Cs]. Retrieved from http://arxiv.org/abs/1805.03677 Holzinger, A., Haibe-Kains, B., & Jurisica, I. (2019). Why imaging data alone is not enough: AI-based integration of imaging, omics, and clinical data. European Journal of Nuclear Medicine and Molecular Imaging. https://doi.org/10.1007/s00259-019-04382-9 Information Commissioner. (2018, June 6). Royal Free-Google DeepMind trial failed to comply with data protection law. Retrieved 15 September 2019, from https://ico.org.uk/about-the-ico/news-andevents/news-and-blogs/2017/07/royal-free-google-deepmind-trial-failed-to-comply-with-dataprotection-law/ Information Commissioner. (2019, June 3). Project ExplAIn interim report. Retrieved 15 September 2019, from https://ico.org.uk/about-the-ico/research-and-reports/project-explain-interim-report/ Jiang, W., & Yin, Z. (2015). Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks. Proceedings of the 23rd ACM International Conference on Multimedia MM '15, 1307– 1310. https://doi.org/10.1145/2733373.2806333 Juengst, E., McGowan, M. L., Fishman, J. R., & Settersten, R. A. (2016). From "Personalized" to "Precision" Medicine: The Ethical and Social Implications of Rhetorical Reform in Genomic Medicine. Hastings Center Report, 46(5), 21–33. https://doi.org/10.1002/hast.614 Kalkman, S., Mostert, M., Gerlinger, C., Van Delden, J. J. M., & Van Thiel, G. J. M. W. (2019). Responsible data sharing in international health research: A systematic review of principles and norms. BMC Medical Ethics, 20(1). https://doi.org/10.1186/s12910-019-0359-9 Kalmady, S. V., Greiner, R., Agrawal, R., Shivakumar, V., Narayanaswamy, J. C., Brown, M. R. G., ... Venkatasubramanian, G. (2019). Towards artificial intelligence in mental health by improving schizophrenia prediction with multiple brain parcellation ensemble-learning. Npj Schizophrenia, 5(1), 2. https://doi.org/10.1038/s41537-018-0070-8 Kim, H., & Xie, B. (2017). Health literacy in the eHealth era: A systematic review of the literature. Patient Education and Counseling, 100(6), 1073–1082. 31 King, D., Karthikesalingam, A., Hughes, C., Montgomery, H., Raine, R., Rees, G., & On behalf of the DeepMind Health Team. (2018). Letter in response to Google DeepMind and healthcare in an age of algorithms. Health and Technology, 8(1), 11–13. https://doi.org/10.1007/s12553-018-0228-4 Kleinpeter, E. (2017). Four Ethical Issues of "E-Health". IRBM, 38(5), 245–249. https://doi.org/10.1016/j.irbm.2017.07.006 Kluge, E.-H., Lacroix, P., & Ruotsalainen, P. (2018). Ethics Certification of Health Information Professionals. Yearbook of Medical Informatics, 27(01), 037–040. https://doi.org/10.1055/s-0038-1641196 Kohli, M., & Geis, R. (2018). Ethics, Artificial Intelligence, and Radiology. Journal of the American College of Radiology, 15(9), 1317–1319. https://doi.org/10.1016/j.jacr.2018.05.020 Kourou, K., Exarchos, T. P., Exarchos, K. P., Karamouzis, M. V., & Fotiadis, D. I. (2015). Machine learning applications in cancer prognosis and prediction. Computational and Structural Biotechnology Journal, 13, 8– 17. https://doi.org/10.1016/j.csbj.2014.11.005 Krutzinna, J., Taddeo, M., & Floridi, L. (2018). Enabling Posthumous Medical Data Donation: An Appeal for the Ethical Utilisation of Personal Health Data. Science and Engineering Ethics. https://doi.org/10.1007/s11948-018-0067-8 Kuek, A., & Hakkennes, S. (2019). Healthcare staff digital literacy levels and their attitudes towards information systems. Health Informatics Journal, 146045821983961. https://doi.org/10.1177/1460458219839613 Kunapuli, G., Varghese, B. A., Ganapathy, P., Desai, B., Cen, S., Aron, M., ... Duddalwar, V. (2018). A Decision-Support Tool for Renal Mass Classification. Journal of Digital Imaging, 31(6), 929–939. https://doi.org/10.1007/s10278-018-0100-0 Kuo, W.-J., Chang, R.-F., Chen, D.-R., & Lee, C. C. (2001). Data mining with decision trees for diagnosis of breast tumor in medical ultrasonic images. Breast Cancer Research and Treatment, 66(1), 51–57. https://doi.org/10.1023/A:1010676701382 Larsen, M. E., Huckvale, K., Nicholas, J., Torous, J., Birrell, L., Li, E., & Reda, B. (2019). Using science to sell apps: Evaluation of mental health app store quality claims. Npj Digital Medicine, 2(1), 18. https://doi.org/10.1038/s41746-019-0093-1 Lee, L. M. (2017). Ethics and subsequent use of electronic health record data. Journal of Biomedical Informatics, 71, 143–146. https://doi.org/10.1016/j.jbi.2017.05.022 Li, B., Li, J., Jiang, Y., & Lan, X. (2019). Experience and reflection from China's Xiangya medical big data project. Journal of Biomedical Informatics, 93. https://doi.org/10.1016/j.jbi.2019.103149 Liu, C., Liu, X., Wu, F., Xie, M., Feng, Y., & Hu, C. (2018). Using Artificial Intelligence (Watson for Oncology) for Treatment Recommendations Amongst Chinese Patients with Lung Cancer: Feasibility Study. Journal of Medical Internet Research, 20(9), e11087. https://doi.org/10.2196/11087 López-Martínez, F., Núñez-Valdez, E. R., Lorduy Gomez, J., & García-Díaz, V. (2019). A neural network approach to predict early neonatal sepsis. Computers & Electrical Engineering, 76, 379–388. https://doi.org/10.1016/j.compeleceng.2019.04.015 Lu, F. S., Hattab, M. W., Clemente, C. L., Biggerstaff, M., & Santillana, M. (2019). Improved state-level influenza nowcasting in the United States leveraging Internet-based data and network approaches. Nature Communications, 10(1). https://doi.org/10.1038/s41467-018-08082-0 Lu, H., & Wang, M. (2019). RL4health: Crowdsourcing Reinforcement Learning for Knee Replacement Pathway Optimization. ArXiv:1906.01407 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1906.01407 Luxton, D. D. (2014). Artificial intelligence in psychological practice: Current and future applications and implications. Professional Psychology: Research and Practice, 45(5), 332–339. https://doi.org/10.1037/a0034559 Maher, N. A., Senders, J. T., Hulsbergen, A. F. C., Lamba, N., Parker, M., Onnela, J.-P., ... Broekman, M. L. D. (2019). Passive data collection and use in healthcare: A systematic review of ethical issues. International Journal of Medical Informatics, 129, 242–247. https://doi.org/10.1016/j.ijmedinf.2019.06.015 Mahomed, S. (2018). Healthcare, artificial intelligence and the Fourth Industrial Revolution: Ethical, social and legal considerations. South African Journal of Bioethics and Law, 11(2), 93. https://doi.org/10.7196/SAJBL.2018.v11i2.00664 Mann, S. P., Savulescu, J., & Sahakian, B. J. (2016). Facilitating the ethical use of health data for the benefit of society: Electronic health records, consent and the duty of easy rescue. Philosophical Transactions of the 32 Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2083). https://doi.org/10.1098/rsta.2016.0130 Marill, M. (2019, May 30). Why Tracking Your Symptoms Can Make You Feel Worse. Wired. Retrieved from https://www.wired.com/story/why-tracking-your-symptoms-can-make-you-feelworse/?&utm_campaign=the_download.unpaid.engagement&utm_source=hs_email&utm_medium =email&utm_content=73227565&_hsenc=p2ANqtz8UIkyITAj5PaGHIR7rQySwYgVDMJAdHABei6CktNIvvIaayTN3lsHJHhDMSWxcl0rGKAf23ckp D3uUAEZow-HQyxjsPJVKPfptCzqqpl2XART5QOA&_hsmi=73227565 Martani, A., Shaw, D., & Elger, B. S. (2019). Stay fit or get bit-Ethical issues in sharing health data with insurers' apps. Swiss Medical Weekly, 149, w20089. https://doi.org/10.4414/smw.2019.20089 Matthias, A. (2015). Robot Lies in Health Care: When Is Deception Morally Permissible? Kennedy Institute of Ethics Journal, 25(2), 169–162. https://doi.org/10.1353/ken.2015.0007 McDougall, R. J. (2019). Computer knows best? The need for value-flexibility in medical AI. Journal of Medical Ethics, 45(3), 156–160. https://doi.org/10.1136/medethics-2018-105118 McLaughlin, K. (2016). Empowerment: A critique. Retrieved from http://public.eblib.com/choice/publicfullrecord.aspx?p=4332655 Millett, S., & O'Leary, P. (2015). Revisiting consent for health information databanks. Research Ethics, 11(3), 151–163. https://doi.org/10.1177/1747016115587964 Millett, Stephan, & O'Leary, P. (2015). Revisiting consent for health information databanks. Research Ethics, 11(3), 151–163. https://doi.org/10.1177/1747016115587964 Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2018). Deep learning for healthcare: Review, opportunities and challenges. Briefings in Bioinformatics, 19(6), 1236–1246. https://doi.org/10.1093/bib/bbx044 Mittelstadt, B. (2019). The Ethics of Biomedical 'Big Data' Analytics. Philosophy & Technology, 32(1), 17–21. https://doi.org/10.1007/s13347-019-00344-z Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016). The ethics of algorithms: Mapping the debate. Big Data & Society, 3(2), 205395171667967. https://doi.org/10.1177/2053951716679679 Morley, J, & Joshi, I. (2019). Developing effective policy to support Artificial Intelligence in Health and Care. Eurohealth, 25(2). https://doi.org/Forthcoming Morley, Jessica, & Floridi, L. (In Press). The need for an ethically mindful approach to AI for healthcare. The Lancet. Morley, Jessica, & Floridi, L. (2019a). Enabling digital health companionship is better than empowerment. The Lancet Digital Health, S2589750019300792. https://doi.org/10.1016/S2589-7500(19)30079-2 Morley, Jessica, & Floridi, L. (2019b). How to design a governable digital health ecosystem. Health Policy and Technology. https://doi.org/10.13140/rg.2.2.28320.74244/1 Morley, Jessica, & Floridi, L. (2019c). The Limits of Empowerment: How to Reframe the Role of mHealth Tools in the Healthcare Ecosystem. Science and Engineering Ethics. https://doi.org/10.1007/s11948019-00115-1 Morley, Jessica, Floridi, L., Kinsey, L., & Elhalal, A. (2019). From What to How. An Overview of AI Ethics Tools, Methods and Research to Translate Principles into Practices. Science and Engineering Ethics. Retrieved from http://arxiv.org/abs/1905.06876 (Pre-Print) Mortazavi, B. J., Desai, N., Zhang, J., Coppi, A., Warner, F., Krumholz, H. M., & Negahban, S. (2017). Prediction of Adverse Events in Patients Undergoing Major Cardiovascular Procedures. IEEE Journal of Biomedical and Health Informatics, 21(6), 1719–1729. https://doi.org/10.1109/JBHI.2017.2675340 Moscoso, A., Silva-Rodríguez, J., Aldrey, J. M., Cortés, J., Fernández-Ferreiro, A., Gómez-Lado, N., ... Aguiar, P. (2019). Prediction of Alzheimer's disease dementia with MRI beyond the short-term: Implications for the design of predictive models. NeuroImage: Clinical, 23, 101837. https://doi.org/10.1016/j.nicl.2019.101837 Nag, N., Pandey, V., Oh, H., & Jain, R. (2017). Cybernetic Health. ArXiv:1705.08514 [Cs]. Retrieved from http://arxiv.org/abs/1705.08514 National Institute for Health Research. (2019, June 14). Involving the public in complex questions around artificial intelligence research. Retrieved 15 September 2019, from https://www.nihr.ac.uk/blog/involving-the-public-in-complex-questions-around-artificialintelligence-research/12236 33 Nebeker, C., Harlow, J., Espinoza Giacinto, R., Orozco-Linares, R., Bloss, C. S., & Weibel, N. (2017). Ethical and regulatory challenges of research using pervasive sensing and other emerging technologies: IRB perspectives. AJOB Empirical Bioethics, 8(4), 266–276. https://doi.org/10.1080/23294515.2017.1403980 Nebeker, C., Torous, J., & Bartlett Ellis, R. J. (2019). Building the case for actionable ethics in digital health research supported by artificial intelligence. BMC Medicine, 17(1). https://doi.org/10.1186/s12916019-1377-7 Nelson, A., Herron, D., Rees, G., & Nachev, P. (2019). Predicting scheduled hospital attendance with artificial intelligence. Npj Digital Medicine, 2(1), 26. https://doi.org/10.1038/s41746-019-0103-3 Ngiam, K. Y., & Khor, I. W. (2019). Big data and machine learning algorithms for health-care delivery. The Lancet Oncology, 20(5), e262–e273. https://doi.org/10.1016/S1470-2045(19)30149-4 NHS England. (2019). The NHS Long Term Plan. Retrieved from NHS website: https://www.longtermplan.nhs.uk/wp-content/uploads/2019/01/nhs-long-term-plan.pdf Nittas, V., Mütsch, M., Ehrler, F., & Puhan, M. A. (2018). Electronic patient-generated health data to facilitate prevention and health promotion: A scoping review protocol. BMJ Open, 8(8). https://doi.org/10.1136/bmjopen-2017-021245 O'Doherty, K. C., Christofides, E., Yen, J., Bentzen, H. B., Burke, W., Hallowell, N., ... Willison, D. J. (2016). If you build it, they will come: Unintended future uses of organised health data collections Donna Dickenson, Sandra Soo-Jin Lee, and Michael Morrison. BMC Medical Ethics, 17(1). https://doi.org/10.1186/s12910-016-0137-x Olimid, A. P., Rogozea, L. M., & Olimid, D.-A. (2018). Ethical approach to the genetic, biometric and health data protection and processing in the new EU general data protection regulation (2018). Romanian Journal of Morphology and Embryology, 59(2), 631–636. Retrieved from https://www.scopus.com/inward/record.uri?eid=2-s2.085052735984&partnerID=40&md5=6b8d7af00f77e58b4439983e09148535 Owens, J., & Cribb, A. (2019). 'My Fitbit Thinks I Can Do Better!' Do Health Promoting Wearable Technologies Support Personal Autonomy? Philosophy & Technology, 32(1), 23–38. https://doi.org/10.1007/s13347-017-0266-2 Page, S. A., Manhas, K. P., & Muruve, D. A. (2016). A survey of patient perspectives on the research use of health information and biospecimens. BMC Medical Ethics, 17(1). https://doi.org/10.1186/s12910016-0130-4 Panch, T., Mattie, H., & Celi, L. A. (2019). The "inconvenient truth" about AI in healthcare. Npj Digital Medicine, 2(1), 77. https://doi.org/10.1038/s41746-019-0155-4 Parker, L., Halter, V., Karliychuk, T., & Grundy, Q. (2019). How private is your mental health app data? An empirical study of mental health app privacy policies and practices. International Journal of Law and Psychiatry, 64, 198–204. https://doi.org/10.1016/j.ijlp.2019.04.002 Petrini, C. (2015). On the 'pendulum' of bioethics. Clinica Terapeutica, 166(2), 82–84. https://doi.org/10.7417/CT.2015.1821 Ploug, T., & Holm, S. (2016). Meta consent-A flexible solution to the problem of secondary use of health data. Bioethics, 30(9), 721–732. https://doi.org/10.1111/bioe.12286 Ploug, Thomas, & Holm, S. (2019). The right to refuse diagnostics and treatment planning by artificial intelligence. Medicine, Health Care, and Philosophy. https://doi.org/10.1007/s11019-019-09912-8 Popkes, A.-L., Overweg, H., Ercole, A., Li, Y., Hernández-Lobato, J. M., Zaykov, Y., & Zhang, C. (2019). Interpretable Outcome Prediction with Sparse Bayesian Neural Networks in Intensive Care. ArXiv:1905.02599 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1905.02599 Powles, J., & Hodson, H. (2017). Google DeepMind and healthcare in an age of algorithms. Health and Technology, 1–17. https://doi.org/10.1007/s12553-017-0179-1 Powles, J., & Hodson, H. (2018). Response to DeepMind. Health and Technology, 8(1), 15–29. https://doi.org/10.1007/s12553-018-0226-6 Price, W. N. (2018). Medical Malpractice and Black-Box Medicine. In I. G. Cohen, H. F. Lynch, E. Vayena, & U. Gasser (Eds.), Big Data, Health Law, and Bioethics (1st ed., pp. 295–306). https://doi.org/10.1017/9781108147972.027 Price, W. N., Gerke, S., & Cohen, I. G. (2019). Potential Liability for Physicians Using Artificial Intelligence. JAMA. https://doi.org/10.1001/jama.2019.15064 34 Quinn, P. (2017). The Anonymisation of Research Data-A Pyric Victory for Privacy that Should Not Be Pushed Too Hard by the EU Data Protection Framework? European Journal of Health Law, 24(4), 347– 367. https://doi.org/10.1163/15718093-12341416 Racine, E., Boehlen, W., & Sample, M. (2019). Healthcare uses of artificial intelligence: Challenges and opportunities for growth. Healthcare Management Forum. https://doi.org/10.1177/0840470419843831 Rahwan, I. (2018). Society-in-the-Loop: Programming the Algorithmic Social Contract. Ethics and Information Technology, 20(1), 5–14. https://doi.org/10.1007/s10676-017-9430-8 Rajkomar, A., Hardt, M., Howell, M. D., Corrado, G., & Chin, M. H. (2018). Ensuring Fairness in Machine Learning to Advance Health Equity. Annals of Internal Medicine, 169(12), 866. https://doi.org/10.7326/M18-1990 Rich, E., & Miah, A. (2014). Understanding Digital Health as Public Pedagogy: A Critical Framework. Societies, 4(2), 296–315. https://doi.org/10.3390/soc4020296 Richardson, V., Milam, S., & Chrysler, D. (2015). Is Sharing De-identified Data Legal? The State of Public Health Confidentiality Laws and Their Interplay with Statistical Disclosure Limitation Techniques. Journal of Law, Medicine and Ethics, 43(s1), 83–86. https://doi.org/10.1111/jlme.12224 Riek, L. D. (2016). Robotics Technology in Mental Health Care. In Artificial Intelligence in Behavioral and Mental Health Care (pp. 185–203). https://doi.org/10.1016/B978-0-12-420248-1.00008-8 Rosenfeld, A., Benrimoh, D., Armstrong, C., Mirchi, N., Langlois-Therrien, T., Rollins, C., ... YanivRosenfeld, A. (2019). Big Data Analytics and AI in Mental Healthcare. ArXiv:1903.12071 [Cs]. Retrieved from http://arxiv.org/abs/1903.12071 Ruckenstein, M., & Schüll, N. D. (2017). The Datafication of Health. Annual Review of Anthropology, 46(1), 261– 278. https://doi.org/10.1146/annurev-anthro-102116-041244 Schönberger, D. (2019). Artificial intelligence in healthcare: A critical analysis of the legal and ethical implications. International Journal of Law and Information Technology, 27(2), 171–203. https://doi.org/10.1093/ijlit/eaz004 Sethi, N., & Laurie, G. T. (2013). Delivering proportionate governance in the era of eHealth: Making linkage and privacy work together. Medical Law International, 13(2–3), 168–204. https://doi.org/10.1177/0968533213508974 Shaw, D. M., Gross, J. V., & Erren, T. C. (2016). Data donation after death: A proposal to prevent the waste of medical research data. EMBO Reports, 17(1), 14–17. https://doi.org/10.15252/embr.201541802 Shickel, B., Tighe, P. J., Bihorac, A., & Rashidi, P. (2018). Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis. IEEE Journal of Biomedical and Health Informatics, 22(5), 1589–1604. https://doi.org/10.1109/JBHI.2017.2767063 Sterckx, S., Rakic, V., Cockbain, J., & Borry, P. (2016). "You hoped we would sleep walk into accepting the collection of our data": Controversies surrounding the UK care.data scheme and their wider relevance for biomedical research. Medicine, Health Care and Philosophy, 19(2), 177–190. https://doi.org/10.1007/s11019-015-9661-6 Sullivan, H. R., & Schweikart, S. J. (2019). Are current tort liability doctrines adequate for addressing injury caused by AI? AMA Journal of Ethics, 21(2), 160–166. https://doi.org/10.1001/amajethics.2019.160 Taddeo, M., & Floridi, L. (2018). How AI can be a force for good: Science, 361(6404), 751–752. https://doi.org/10.1126/science.aat5991 Topol, E. J. (2019). High-performance medicine: The convergence of human and artificial intelligence. Nature Medicine, 25(1), 44–56. https://doi.org/10.1038/s41591-018-0300-7 Topuz, K., Zengul, F. D., Dag, A., Almehmi, A., & Yildirim, M. B. (2018). Predicting graft survival among kidney transplant recipients: A Bayesian decision support model. Decision Support Systems, 106, 97–109. https://doi.org/10.1016/j.dss.2017.12.004 Townend, D. (2018). Conclusion: Harmonisation in genomic and health data sharing for research: An impossible dream? Human Genetics, 137(8), 657–664. https://doi.org/10.1007/s00439-018-1924-x Tran, B. X., Vu, G. T., Ha, G. H., Vuong, Q.-H., Ho, M.-T., Vuong, T.-T., ... Ho, R. C. M. (2019). Global Evolution of Research in Artificial Intelligence in Health and Medicine: A Bibliometric Study. Journal of Clinical Medicine, 8(3). https://doi.org/10.3390/jcm8030360 Turilli, M., & Floridi, L. (2009). The ethics of information transparency. Ethics and Information Technology, 11(2), 105–112. https://doi.org/10.1007/s10676-009-9187-9 35 Vayena, E., Tobias, H., Afua, A., & Allesandro, B. (2018). Digital health: Meeting the ethical and policy challenges. Swiss Medical Weekly, 148(34). https://doi.org/10.4414/smw.2018.14571 Vayena, Effy, Blasimme, A., & Cohen, I. G. (2018). Machine learning in medicine: Addressing ethical challenges. PLoS Medicine, 15(11), e1002689. https://doi.org/10.1371/journal.pmed.1002689 Vayena, Effy, Salathé, M., Madoff, L. C., & Brownstein, J. S. (2015). Ethical Challenges of Big Data in Public Health. PLOS Computational Biology, 11(2), e1003904. https://doi.org/10.1371/journal.pcbi.1003904 Voigt, K. (2019). Social Justice, Equality and Primary Care: (How) Can 'Big Data' Help? Philosophy & Technology, 32(1), 57–68. https://doi.org/10.1007/s13347-017-0270-6 Vollmer, S., Mateen, B. A., Bohner, G., Király, F. J., & Ghani, R. (2018). Machine learning and AI research for Patient Benefit: 20 Critical Questions on Transparency, Replicability, Ethics and Effectiveness. 25. Vollmer, S., Mateen, B. A., Bohner, G., Király, F. J., Ghani, R., Jonsson, P., ... Hemingway, H. (2018). Machine learning and AI research for Patient Benefit: 20 Critical Questions on Transparency, Replicability, Ethics and Effectiveness. ArXiv:1812.10404 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1812.10404 Wachter, R. M. (2015). The digital doctor: Hope, hype, and harm at the dawn of medicine's computer age. New York: McGraw-Hill Education. Wang, Z. (2019). Data integration of electronic medical record under administrative decentralization of medical insurance and healthcare in China: A case study. Israel Journal of Health Policy Research, 8(1). https://doi.org/10.1186/s13584-019-0293-9 Wardrope, A. (2015). Relational Autonomy and the Ethics of Health Promotion. Public Health Ethics, 8(1), 50– 62. https://doi.org/10.1093/phe/phu025 Watson, D. S., Krutzinna, J., Bruce, I. N., Griffiths, C. E., McInnes, I. B., Barnes, M. R., & Floridi, L. (2019). Clinical applications of machine learning algorithms: Beyond the black box. BMJ, 364, l886. https://doi.org/10.1136/bmj.l886 World Health Organisation. (n.d.). Big data and artificial intelligence. Retrieved 29 June 2019, from https://www.who.int/ethics/topics/big-data-artificial-intelligence/en/ Zacher, B., & Czogiel, I. (2019). Supervised learning improves disease outbreak detection. ArXiv:1902.10061 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1902.10061 Zhang, L., Wang, H., Li, Q., Zhao, M.-H., & Zhan, Q.-M. (2018). Big data and medical research in China. BMJ, j5910. https://doi.org/10.1136/bmj.j5910 View publication stats