Disruptive Innovation and Moral Uncertainty∗ Accepted for publication in NanoEthics: Studies of New and Emerging Technologies. Preprint manuscript version of 28 August 2020. Philip J. Nickel, Eindhoven University of Technology, p.j.nickel@tue.nl Abstract: This paper develops a philosophical account of moral disruption. According to Robert Baker (2013), moral disruption is a process in which technological innovations undermine established moral norms without clearly leading to a new set of norms. Here I analyze this process in terms of moral uncertainty, formulating a philosophical account with two variants. On the Harm Account, such uncertainty is always harmful because it blocks our knowledge of our own and others' moral obligations. On the Qualified Harm Account, there is no harm in cases where moral uncertainty is related to innovation that is "for the best" in historical perspective, or where uncertainty is the expression of a deliberative virtue. The two accounts are compared by applying them to Baker's historical case of the introduction of mechanical ventilation and organ transplantation technologies, as well as the present-day case of mass data practices in the health domain. Keywords: disruptive innovation, disruptive technology, moral disruption, moral uncertainty, withdrawal of life support, ethics of organ transplantation, data ethics I. Introduction In his book Before Bioethics, Robert Baker defines morally disruptive technological innovations as those which "undermine established moral norms or ethical codes" (Baker 2013, 59).1 Baker's definition refers to situations in which the established collective moral norms associated with a set of practices are undermined through technological innovation, without new moral norms clearly emerging. This paper explores the nature of such disruption and its impact on individual moral agents. It is hoped that by better understanding the moral impact of technological disruption, we may obtain a new lens for interpreting social and political reactions to disruptive technologies. I shall examine the view that in situations of moral disruption, people are made morally worse off in the respect that they are caused not to know their own moral obligations, or those of others (at least not determinately).2 Let us label this the moral disruption as harm account, or simply the Harm * This paper has benefited from the comments of two anonymous reviewers, as well as from an audience at Eindhoven University of Technology and at conferences at the University of Warwick and the Universidade Nova de Lisboa. I particularly thank Gunter Bombaerts and Julia Hermann for their written comments on an earlier manuscript. 1 Disruptive innovations are defined in terms of their powerful effects on existing products and business models (Christensen 1997; Markides 2006). Morally disruptive technologies usually fall within this class, having powerful non-moral effects as well as moral effects. 2 This analysis concerns uncertain moral norms and obligations: in this respect it is distinct from the typical analysis of uncertainty in relation to emerging technologies in terms of risk, or in terms of future outcomes about which we are ignorant (Brey 2017, Hansson 2013). 2 Account. In principle, such an account is relevant to any type of moral disruption, not merely that caused by technological developments. Here, however, I will confine the discussion to cases of technological innovation. In order to determine whether such an account is sound, I examine Baker's own case. The introduction of mechanical ventilation technologies in the mid-Twentieth Century led to situations in which individuals who would never regain minimal function were kept alive for long periods of time. This created moral uncertainty about the appropriate regard for these people's bodies. In addition, by increasing opportunities for effective organ donation, it opened up the possibility to use "brain dead" bodies as a life-saving resource, creating further ambiguity about the moral status of these bodies and their parts. This case raises a difficulty for the Harm Account. When disruption turns out for the best, as this case seems to have done - in the sense that it transforms our moral practices in such a way that the technological innovation is highly morally good overall (however we wish to unpack this idea) - the account nonetheless implies that it makes individuals worse off for a particular timespan in one respect. This seems bizarre: in cases where changing the norm is a good thing, how can moving from the old norm to a state of uncertainty about it be bad? In addition, uncertainty seems, at least in some people, to be part and parcel of virtuous deliberation about how to respond to a difficult situation. Why should the exemplification of virtue count as a harm? These implications may prompt us to qualify the Harm Account by stipulating that it only applies to situations in which moral disruption is not linked to moral progress, and when it does not lead to the exemplification of virtuous dispositions. Let us label this the Qualified Harm Account. According to the Qualified Harm Account, the moral uncertainty associated with mechanical ventilation technologies is harmful except when one of these qualifications applies. How do the two accounts - the Harm Account and the Qualified Harm Account - apply to present-day cases of moral disruption? By definition, any present-day case of moral disruption will involve a live, unresolved question about whether undermining existing norms will turn out to be a good thing. There will often be disagreement about the change of norms, with some voices proclaiming a strong position in favor of a new set of norms, and some voices proclaiming attachment to the old set. The present-day case of mass health data practices - the systematic and widespread collection and processing of health data for purposes of health promotion, science, and commerce - exhibits moral uncertainty. With this present-day case in mind, in section IV I will compare the two accounts. Surprisingly, despite their differences they give similar guidance. This paper does not favor one account over the other. Since these matters have not been treated in earlier literature (at least not in this guise), the purpose of this paper is to develop the topic for further inquiry. II. Moral disruption as harm People do not like being uncertain. Psychologists have identified a pattern of "ambiguity aversion" in which people prefer options involving known chances over options in which the 3 chances cannot be estimated (Ellsberg 1961).3 Although the dislike of uncertainty is widely shared, there are measurable differences in people's tolerance of uncertainty. Some people feel it more strongly than others (Berenbaum et al 2008, Freeston et al. 1994). One reason we might regard moral uncertainty as troubling is the mere fact that it makes so many people dissatisfied. However, as a way of understanding the adverse effects of moral uncertainty, ambiguity aversion does not get us very far. Psychological feelings of dissatisfaction with moral uncertainty could not themselves constitute a harm, because neither (a) uncertainty nor (b) harm is a mere psychological state. Ad (a): Uncertainty in its proper sense is not just a psychological state, but also an epistemic state in which a person has insufficient reason to close a given question. Psychological feelings of uncertainty are normally responsive to a lack of sufficient reasons, and it is this epistemic insufficiency of reasons that partly constitutes uncertainty. The condition in which a person feels uncertain in response to a genuine inability to close an important moral question can be called grounded moral uncertainty. Ad (b): Harm is also not a psychological state, but (on an influential reading) a condition in which a person's interests have been seriously set back (Feinberg 1984, 36). The felt effects of uncertainty constitute disutility, not harm. To see them as a harm is to get things backwards. The disutility of feelings associated with moral attitudes is a response to a harm, rather than a harm in itself. Grief, shame, and betrayal, for example, are not harmful because of the displeasure they involve. They involve displeasure as an apt response to perceived harms or other wrongs. These harms or wrongs are prior to the displeasure. Because ambiguity aversion is a psychological notion, focusing on it alone therefore cannot address the problem of moral uncertainty. A philosophical interpretation is needed. The line of interpretation to be pursued here is that moral uncertainty constitutes a harm because a person is thereby in a condition of not knowing her own moral obligations (rights, responsibilities) or the obligations (rights, responsibilities) of others. This constitutes a harm because an individual who does not know her own moral obligations (rights, responsibilities) cannot properly exercise moral agency or act rightly, which is a serious setback to her interests as a moral agent. She also cannot properly form moral expectations of others or hold others morally accountable for their actions. This is the basis of the Harm Account. The Harm Account does not derive from or commit itself to just a single type of normative moral theory. A deontological theory of moral agency, on which one acts well by being responsive to one's moral duties as settled upon by a community of reasonable persons, leads rather naturally to the idea that moral uncertainty is disruptive of agency. A consequentialist theory would also logically hold that a person who acts in a state of 3 Bear in mind that uncertainty is being contrasted with risk here. The latter is quantifiable uncertainty. So, although there are many exceptions to this generalization, risk-seeking (where the chances are predictable) is not strictly one of them. 4 uncertainty about the good is more likely to produce the wrong outcome than a person who acts in a state of determinate knowledge of the good. It may also take account of the disutility of lacking clear standards for holding others responsible and being held responsible. In these ways, it also sees moral uncertainty as harmful. Virtue ethics also could be taken to support the Harm Account, in cases where disruption is an impediment to the display of virtues of deliberation and right action.4 In these various ways, the Harm Account can be used to develop what Baker says about the moral disruption caused by mechanical ventilation. Mechanical ventilation is the use of pumped air to aid in breathing when the lungs are not able to pump on their own. In the mid Twentieth Century it began to be applied effectively for temporary assistance of breathing function in humans, originally using an "iron lung," a pressure-controlled box inside which the patient was placed, or a hand pump. With greater medical knowledge of respiration and lung function, the pressure box could be abandoned and the pump could be automated for a longer duration of application, while preventing lung damage and adverse effects on blood gases (Slutsky 2015). By the 1960s, some people with severe brain injury could be kept alive indefinitely using mechanical ventilation. For Baker, this is what created moral disruption: [T]he ventilator kept some patient's hearts and lungs functioning even though they appeared to be in an irreversibly comatose state, with little or no brain function. This physiological phenomenon proved morally and ethically disruptive because medical ethics and morality mandated, to quote the New York [physician's] oath, that physicians 'do everything in [their] power for the benefit of the sick committed to [their] charge,' and this mandate was understood as preserving life. ... [N]either physicians nor families now knew how to treat patients in this ambiguous state. (Baker 2013, 59). According to Baker's description, moral disruption can be seen here in the moral uncertainty of professionals and family members about whether it is permissible to withdraw ventilation in a person with little or no brain function. Applying the Harm Account, the harmful effects to focus on are the fact that professionals did not know what their own moral obligations and permissions were, and that families not only did not know their own moral obligations, but also did not know what standards to use in deciding whether to trust physicians in this situation. The familiar basis for moral agency was missing.5 A further effect of mechanical ventilation, also mentioned by Baker, was to make the bodies of "brain dead" patients available for organ transplantation. Kept in a state where their organs were alive, but their brains were not functioning, these patients were potentially useful as sources of donated organs. This possibility further amplified the moral uncertainty 4 There are normative theories in the traditions of Socratic and Pyrrhonian skepticism according to which a state of suspended judgment is beneficial and should be strived for. But only the most radical versions would propose that lacking a basis for action is itself beneficial. 5 In more recent work (Baker 2019), Baker emphasizes the role that Kuhnian paradigm shifts play in the production of public controversy. Although Baker still admits that "paradigms become uncertain" (176), the emphasis of the more recent work is on the dissident nature of the paradigm shift, rather than the ambiguous underlying epistemic state. 5 on the part of professionals and family members by raising the stakes. Was the loved one's body to be treated in the usual way that a person would be, or permissibly mined for a lifesaving resource, the vital organs themselves? There are thus two somewhat distinct clouds of ambiguity involved in the mechanical ventilation case. The first is linked to a descriptive ambiguity in the status of the person needing ventilation: whether she is alive, dead, or (perhaps puzzlingly) "brain dead". Along with this descriptive ambiguity come a host of relevant moral ambiguities such as whether removing ventilation constitutes a killing or an instance of "letting die", and (more prosaically) whether it is appropriate for family members to mourn a death. The second cloud of ambiguity is linked to the status of the body as a valued good and is somewhat independent of whether it is dead or alive. Is it appropriate to use body parts as a fungible resource, and how? Do they have absolutely non-quantifiable value ("dignity" in Kantian terms), or can they be transferred as a gift or even an item of sale?6 This second cloud of ambiguity has historical links with earlier (highly controversial and often illegal) practices of Anglo-American medical schools obtaining dead bodies - through donation, sale, and even grave-robbing - for purposes of medical education and experimentation (Richardson 2006).7 On the Harm Account, then, the setback to interest in the case of mechanical ventilation consists of an inability to know one's own obligations and the obligations of others regarding withdrawal of ventilation and questions of organ transplantation and use. This uncertainty ramifies to effects involving trust and commitment, since a morally uncertain person does not know what standards and expectations to hold others to in relying on them, nor what others can reasonably trust them to do. There are other downstream effects involving other moral attitudes such as guilt, blame, betrayal, and regret: should I feel guilty for a decision I made without an adequate basis? When my medical doctors put me in this situation of moral uncertainty, have they caused me harm? Should I blame them? The uncertainty here gives rise to a kind of moral "dyscrasia" or dysfunction, a state of disorientation analogous to a loss of the sense of direction when trying to navigate. In turn, this moral dyscrasia blocks a person from being able to act (or "navigate") effectively.8 The moral uncertainty under discussion is itself a social phenomenon, transcending the level of individual psychological explanation. The activation of moral attitudes in social context is 6 The commodification of organ transfers remains controversial. Although it is not generally permitted, one pair of authors write, "The gift the recipient has received from the donor is so extraordinary that it is inherently unreciprocal. It has no physical or symbolic equivalent. As a consequence, the giver, the receiver, and their families may find themselves locked in a creditor-debtor-vise that binds them to each other in a mutually fettering way" (Fox and Swazey 2002, 40 [quoted in Scheper-Hughes 2007]). An exchange of money might actually help, in this situation, to release the receiver from this "vise". Others have argued for the social benefits of a market in organs (Harris & Erin 2002). 7 In an early U.S. case in which a legal claim was successfully made against those who had obtained and stored tissue samples from a cadaver for autopsy, Palenzke v. Bruning [1900], the court held that "no-one had the right to remove parts of the body and, without the parent's consent, throw them into a privy vault. Such conduct violates every instinct of propriety, and could not fail to outrage the feelings of the kindred of the deceased." 98 Ill. App. 644 (1900). Referenced in Hamilton (2012): 203. 8 I thank an anonymous reviewer for suggesting this point. 6 part of what sets cases of moral disruption apart from the (presumably temporary and voluntary) moral uncertainty that some students experience in introductory courses on moral philosophy, which we do not typically regard as harmful. Adoption and abandonment of social norms are part of the core phenomenon of moral disruption as Baker describes it: we are in a condition of moral disruption together. In such a condition, people do things that will turn out to have been morally wrong in historical perspective and for which they may be judged later. Even if a person is not blameworthy in the final analysis, their own conscience may also raise persistent questions about whether they should have acted differently. On the Harm Account, then, people do not simply dislike moral uncertainty. They have good reason to feel that it undermines their moral agency.9 III. Two problems for the Harm Account Two problems will now be raised for the Harm Account, which will lead us to formulate an alternative view. The first relates to our historical perspective on moral disruption. The medical community and the public now widely regard it as permissible to withdraw mechanical ventilation from a person who will never regain brain function and to use the person's organs for transplantation. Practices surrounding mechanical ventilation and transplantation have stabilized in a way that has saved many lives. The overall effect of introducing mechanical ventilation, including our revised norms about the treatment of dead bodies, seems to have been morally beneficial. Present-day norms presuppose that there is conclusive reason to favor allowing withdrawal of mechanical ventilation and organ transplantation with the consultation and permission of family. Not all uncertainty has vanished, since families must still make a decision that they might find ambiguous or difficult. But few would condemn professionals who withdraw such life support with the permission of the family or who suggest organ donation, and few would condemn families who go along with these courses of action.10 The first problem is thus that uncertainty about withdrawing mechanical ventilation and organ transplantation appears in historical perspective to be a good thing. In a case where 9 It is interesting to compare moral dilemmas as a putative harm. Perhaps a person who, through no fault of their own, is placed in a sufficiently serious moral dilemma has their interests set back. Moral uncertainty overlaps with but is not identical with the concept of a moral dilemma in this sense. In a moral dilemma, one can have a crystal-clear epistemic grasp of conflicting norms or considerations. In such a state one has uncertainty about how to reconcile conflicting norms or considerations against one another, but not uncertainty about the norms or considerations themselves. Conversely, a state of moral uncertainty can arise without some of the essential features of a moral dilemma: it need not involve an experience of conflicting norms or considerations. In the era when effective mechanical ventilation was introduced, one might have faced uncertainty about whether it would be permissible to authorize the use of a brain dead loved one's organs for transplantation, without thereby apprehending that there were two valid yet conflicting moral norms or considerations in play. Interestingly, Walzer briefly discusses a view he ascribes to Merleau-Ponty according to which "the agony and the guilt feelings experienced by the man who makes a 'dirty hands' decision derive from his radical uncertainty about the actual outcome" (Walzer 1974, 74n.). Walzer's comment that "the anxiety of the gambler is of no great moral interest" (ibid.) shows that he is understanding the relevant uncertainty in a shallow psychological sense. 10 There are still wrenching dilemmas and uncertainties connected to organ transplantation, particularly for people in relative poverty who are in a position to sell a kidney to pay debt or finance a new enterprise (Scheper-Hughes 2007). 7 changing the norm is a good thing, how can moving from the old, bad norms to a state of uncertainty about them be a setback to interests? On the contrary, the uncertainty represents progress and improvement, advancing the person's interests by loosening her attachment to the old norms. The second problem is that it seems that uncertainty can be the active response of a person who is morally serious, careful, and engaged: a deliberatively virtuous person. Some of the uncertainty following the introduction of mechanical ventilation regarding the question of whether to permit withdrawal of ventilation and organ transplantation exhibited such virtues. Uncertainty of this type is intrinsically valuable insofar as these virtues constitute one of the central goods of a deliberative moral community. It is also instrumentally valuable because engaging in deliberation and reflection is part of what leads individuals and societies toward better ways of responding to new situations: it is strongly conducive to socially beneficial practices surrounding new technologies. To call this harm is strange. Roughly speaking, the first problem arises from a consequentialist intuition: disruption that brings moral progress cannot itself be regarded as bad. The second problem arises from a virtue-ethical intuition: the exemplification of deliberative virtues cannot itself be regarded as harmful. These two problems are logically independent of one another but may be mutually reinforcing. Virtuous uncertainty may be conducive to moral progress, or it may have such progress as its objective. Facing these problems, we might be tempted by the alternative account mentioned in Section I, the Qualified Harm Account, which counts some but not all moral disruption as harmful. The first exception to the harmfulness of moral disruption is the situation in which it represents a "letting go" of old norms so as to achieve substantially improved community moral norms. From our historical perspective, the old norms of never withdrawing lifesustaining aid, and never using a living body as a source of organs, were worth abandoning. The Qualified Harm Account does not count uncertainty about them as a harm. The second exception is the situation in which moral disruption is bound together with virtuous moral deliberation about a difficult situation. On the Qualified Harm Account, harmful cases of moral disruption are those where it does not represent progress and improvement, as well as those where it does not provide an opportunity for virtuous practical deliberation. Perhaps the emergence of a practice of dueling with pistols (instead of swords) provides examples of both: the introduction of the practice might cause moral uncertainty about whether to engage in a duel but does not represent moral progress. In addition, it does not exemplify deliberative virtue to be morally uncertain or to puzzle about whether to engage in such a duel (cf. Appiah 2018). The idea that moral progress disqualifies the harm of moral disruption carries metaphysical commitments. It is often only in historical perspective that we draw a conclusion about whether our norms have improved. The Qualified Harm Account cannot say that how things turn out historically somehow determines the status of our earlier uncertainty through a kind of time-reversed causation. The Qualified Harm Account might, however, hold that there can already be a fact of the matter at an earlier point in time about whether the "old" norms should be abandoned. Our historical perspective reveals this fact. 8 Such a story works best on the assumption of technological determinism, according to which the introduction of a technology has an unmediated knock-on effect on our practices. The claim that mechanical ventilation is already "for the best" at some point during the period of moral uncertainty is easier to defend if there is no contingency in how the technology will fit with or be influenced by the practices that emerge along with it. However, this may not be a plausible assumption. The widely-observed phenomenon of technological multistability - in which a technology can develop in many ways depending on how it is framed and embedded in human practices (Carlsen et al. 2010; Rosenberger 2014) - seems to imply that it is not already determined at the time when the technology is first introduced whether the disruption and uncertainty that result are "for the best". As a result, the Qualified Harm Account holds that moral uncertainty often starts out harmful, because there is not yet a fact of the matter about whether the disruption is "for the best," and becomes non-harmful when this fact of the matter is fixed. (This could be sudden or gradual.) This is a strange feature, but one must remember that the Harm Account raises its own problems by painting all moral uncertainty with the same brush. Figure 1. Moral disruption and the harm of uncertainty Figure 1 presents this account schematically. In the situation depicted in the figure, the moral disruption associated with a new technology begins somewhat after that technology has been introduced. (We can compare the two accounts in relation to the figure as follows. On the Harm Account, the full duration (c) of the uncertainty counts as a harm to those who experience it. On the Qualified Harm Account, the harm of moral disruption extends only through time period (a). (a) refers to that period of uncertainty that is not (yet) associated with moral progress. (b) refers to the period of uncertainty where it is determinately the case that the practice has improved. Let us now turn to the second, virtue-ethical qualification of harm. Suppose a physician in 1975 deliberated virtuously about the moral question of whether to permit withdrawal of 9 mechanical ventilation. It seems strange to call his or her state of uncertainty harmful. By contrast, suppose the family of a teenaged victim of a motorcycle accident were forced to deliberate in 1975 about whether to withdraw mechanical ventilation. Even if the family's uncertainty were deliberatively virtuous, exemplifying care and sensitivity, it is not strange to say that the uncertainty is harmful. More generally, we do not think that people are benefited by being provided with opportunities to exemplify the virtues, if what this means is to place them in morally challenging situations without their active participation (cf. Nyholm 2016). If I force you to exemplify courage by placing you in a dangerous situation, I have not made you better off, even though it gave you an opportunity to be morally virtuous throughout. Similarly with the deliberative virtues. IV. Application of the two accounts By considering a present-day case of moral disruption, we can throw the application of the two accounts into better relief. Consider the case of mass data practices for the sake of health and wellness. Currently, practices of mass collection, combination, and analysis of health data on individuals are being adopted in many contexts. Masses of data from explicitly medical contexts, such as patient dossiers and diagnostic scans, are being shared with many private partners for research, therapeutic, and institutional ends. They are also being combined with data from other contexts such as health software applications for the sake of ends such as monitoring and guiding people with chronic illness, and early identification of health risks. Let us suppose, for the sake of argument, that it is currently impossible to know whether changing our norms surrounding personal health information to allow mass data practices would be a good thing. The impact on health outcomes might be beneficial on average, but there may also be significant injustices, privacy violations, and security breaches carried along with such practices. The involvement of significant commercial interests clouds the issue whether mass data practices are a matter of public benefit and the advancement of medical science, or a matter of commodifying private information (Mittelstadt & Floridi 2015). Vulnerable or socially marginalized people may risk being classified in ways that are incorrect or unfair simply because there is less data available about them as a class, and they may be further marginalized because they are irrelevant to commercial interests, or worse, seen as exploitable by them (Iliadis & Russo 2016; Madden et al. 2017). It is then unclear whether each person has an obligation to share their data along with others, according to the logic of solidarity and the pursuit of better medical science, or whether some (or all) are permitted to resist. People are put in the position of needing to decide individually whether to share personal data on many occasions and in many contexts, and to decide collectively whether to embrace mass data collection and processing; but they do not know their own and others' obligations in this domain. They are in a situation of moral disruption as Baker defined it. Elsewhere I have argued that such ambiguities of moral status and resulting moral uncertainties are endemic to contemporary practices of data collection and processing (Nickel 2019). The Harm Account and the Qualified Harm Account, while delivering different theoretical judgments about when and how moral disruption gives rise to a harm, offer surprisingly 10 similar guidance in this case. The Harm Account concludes that there is a definite harm associated with our lack of determinate knowledge of our own and others' moral obligations, for example in regard to whether one ought to share health data, or whether something is owed in return for health data sharing. This implies that we ought to prevent, mitigate, or compensate for the harm where possible. The Qualified Harm Account also registers the possibility of harm in one of several ways. First, some disruption causes harm before the moment when the technology is determinately on the way to producing moral improvement, or outside of situations where it is a welcome opportunity to display deliberative virtues. Second, there is a serious risk of harm through moral disruption, pending the historical outcome. Even if all or part of the moral uncertainty does not cause a harm, in our current state of knowledge about the introduction of the new technology, we do not know whether that is the case. We are in a condition of moral uncertainty. Creating a risk of harm also carries obligations to mitigate the risk or take steps to reduce it. In this way, both the Harm Account and the Qualified Harm Account advise taking steps to prevent, mitigate, or compensate for moral disruption. The two views often overlap in their practical guidance for managing technological disruption. In practice, however, there will often be significant differences in how the two views are applied. Advocates of the Qualified Harm Account who are personally convinced that the moral disruption of mass health data practices is for the best in the long run, or who themselves welcome the opportunity to deliberate about it, are likely to discount the significance of disruption even while there is still widespread moral uncertainty. By contrast, advocates of the Harm Account will take moral disruption seriously as a harm regardless. In practice, it may be necessary to remind those committed to the Qualified Harm Account that they must also take the moral impacts and risks of such a case seriously. V. Do meta-principles for moral uncertainty help? Before concluding, I consider an objection arising from recent philosophical work on moral uncertainty. According to some philosophers, in cases where we do not know the moral value of our actions determinately, or the various possible ways of valuing actions, we can still calculate our moral meta-obligations (Lockhart 2000; Zimmerman 2008, 38; Barry & Tomlin 2016; Lazar 2018; Hansson 2013; Macaskill 2016). The strategy for determining one's meta-obligations is to consider every plausible valuation of one's available actions, and then find a meta-principle to calculate one's unique meta-obligation given these valuations. To take a simple example, suppose the available actions are withdraw mechanical ventilation and do not withdraw mechanical ventilation, and suppose each of these two options could possibly be valued as permissible or impermissible. There are then three possible combinations: permissible-permissible; permissible-impermissible; and impermissiblepermissible. (The combination impermissible-impermissible is ruled out logically on the assumption that this is not a "moral blind alley" case where there is no morally right action (Nagel 1993).) With these three possible valuations, then, we might adopt the permissive meta-principle: "In cases where for all we know, all of the options are permissible in at least 11 one possible (first-order) valuation and impermissible in at least one possible valuation, then all of the options are permissible at the meta-level."11 This is displayed in Figure 2. Poss. Valuation 1 Poss. Valuation 2 Poss. Valuation 3 Meta-valuation Withdraw mechanical ventilation Permissible Permissible Impermissible Permissible Do not withdraw mechanical ventilation Permissible Impermissible Permissible Permissible Figure 2. Calculating meta-valuations in situations of moral uncertainty Some of the moral meta-principles discussed in the philosophical literature take numerical first-order valuations as their inputs (following a consequentialist quantification of outcomes), instead of deontic statuses. For numerical valuations, various mathematical formulae have been proposed for calculating the meta-obligations. Whether the meta-principles are deontological or quantitative, they would appear to raise a problem for both the Harm Account and the Qualified Harm Account, because these two accounts locate the harm of moral uncertainty in the fact that one does not know one's own obligations (rights, responsibilities). But if some such meta-principle is correct, then this thesis is false. Even when one does not know one's determinate obligations, one can know one's meta-obligations, and this is sufficient for moral agency. Let's take a step back. Even if we grant that such meta-principles can be adequately established and calculated, it could still be the case that people who do not have determinate knowledge of their first-order obligations are worse off than those who do have such knowledge, because their moral agency is less adequate as a result or because their relying on the meta-principles carries serious adverse consequence for them. The literature on ambiguity aversion supports such an idea, since people do not like moral uncertainty. Furthermore, meta-principles for moral uncertainty are not currently embedded in current moral practices. As a consequence, even if people rely on them, they are likely to find themselves unsupported by others in their moral reasoning and may be judged by others to have behaved wrongly, or at least to have reasoned wrongly. There are serious costs for individuals. At the present time, we do not have any reliable collective grasp of such meta-obligations and it is not clear if any set of them can be defended sufficiently. The meta-obligations set out and defended in the literature on moral uncertainty are diverse and contradictory. No consistent set of them has been widely agreed upon within the group of philosophers discussing them, still less beyond that group. Such meta-obligations are, quite simply, more uncertain than the possible first-order obligations whose valuations serve as their inputs. As a consequence, introducing them does not reduce or resolve moral uncertainty, nor the harm involved in moral disruption, in any practical sense. 11 Hansson writes that "Uncertainty tends to reduce moral requirements and thereby increase the moral leeway, or moral latitude, so that a larger selection of permissible alternatives is open to the agent" (2013, 81). Although he is not writing specifically about moral uncertainty, the underlying idea of this meta-principle is similar. 12 VI. Conclusion Technological disruption provokes many reactions. The tolerance of uncertainty varies considerably between people. Those who do experience distress (the uncertaintyintolerant) vary in how they respond (Berenbaum et al. 2008). Some with a high threshold for psychological distress might admit feeling uncertain; but they may use coping strategies to frame it in ways that blunt its impact, as Fox has documented in the case of medical professionals (Fox 2000). We might speculate that others avoid feelings of distress related to uncertainty by avowing allegiance to one set of norms or another. Some "conservatives" will respond to situations where moral norms are contested or ambiguous by cleaving to old norms. Other "progressives" will respond by peremptorily adopting new norms. A fortiori, neither reaction is privileged over the other. If this story is right, then the public face of moral uncertainty will often be that of disagreement and controversy, rather than perplexity. The idea of moral disruption developed here only applies strictly to those who actually feel uncertain in the face of moral ambiguity, and the communities to which they belong. However, if actual reactions to situations of moral uncertainty are diverse, then we should be on the lookout, not just for individuals experiencing uncertainty, but for persistent collective disagreement and controversy that shows an ongoing failure to cope with morally disruptive changes. Supposing mechanical ventilation and health data practices are typical cases of moral disruption, it might be wise to be alert for situations in which there is a pervasive ambiguity regarding the moral status of items of value such as personal health data and body parts. One of the axioms of Robert Baker's (2013) book is that it is possible to avoid persistent moral disruption by addressing moral controversies proactively, establishing professional norms and policies that reduce ambiguity. According to Baker, the field of bioethics originated when the American Medical Association failed to deal proactively with the moral uncertainty surrounding mechanical ventilation and the allocation of kidney dialysis, whereupon forces outside the profession felt the need to take over the process of moral deliberation. Many disruptive innovations cannot be dealt with professionally, however, since there is no authoritative, cohesive, and trusted professional community in a position to respond to them. In such cases we will need to seek other ways to mitigating the (possible) harmful effects of technological disruption. Taking our cue from the theory of responsiveness to highly uncertain hazards, we should think about strategies of resilience, hazard reduction, and stable but flexible policymaking (Nickel 2019). A first step is to acknowledge that new technologies create moral uncertainty, and that this is not just an annoyance, but a serious impediment to individual moral agency. There is an important interrelationship between the ethics of new and emerging technologies and the ethics of moral disruption. For example, nanoscale technologies have generated moral controversy and disruption regarding the question of whether they should be introduced in a widespread and uncontrolled way. Technological optimists take the view that we are morally permitted, or even morally obligated, to pursue nanoscale technologies 13 as solutions to such varied problems as global warming and security. Technological pessimists take the opposing view that we are morally forbidden from doing so because of the non-moral, technological and factual, uncertainties and risks associated with nanoscale technologies. An important lesson for thinking about the ethics of moral disruption is that it is often strongly associated with uncertainties regarding what scenarios we should be thinking about and how seriously we should take them. The ethics of risk and uncertainty is far from being a settled matter. This is one reason why moral disruption cleaves so closely to new and emerging technologies. Conversely, it is fruitful to consider the possibility that the controversy surrounding new and emerging technologies can sometimes arise from underlying moral uncertainty. Although the public debate around such technologies may seem polarized, pitting "conservatives" and "progressives" against one another, we must remember that this could be a manifestation of an unsettled and ambiguous epistemic state. Being in such a state is sometimes harmful, blocking people from knowing their own moral obligations and the obligations of others. It is no wonder that people might try to escape this harm by adopting a strong conviction, however epistemically tenuous! In such cases we should try to see through the controversy to identify the underlying harms; to respond with compassion; and to mitigate the harm where possible by creating stability and dialogue. References: Appiah, K.A. 2011. The Honor Code: How Moral Revolutions Happen. W.W. Norton & Co. Baker, R. 2013. Before Bioethics. Oxford University Press. Baker, R. 2019. The Structure of Moral Revolutions: Studies of Changes in the Morality of Abortion, Death, and the Bioethics Revolution. MIT Press. Barry, C., & Tomlin, P. 2016. Moral uncertainty and permissibility: evaluating option sets. Canadian Journal of Philosophy 46: 898–923. Berenbaum, H., Bredemeier, K., & Thompson, R.J. 2008. Intolerance of uncertainty: exploring its dimensionality and associations with need for cognitive closure, psychopathology, and personality. Journal of Anxiety Disorders 22: 117–125. Brey, P. 2017. Ethics of emerging technology. In S.O. Hansson, ed., The Ethics of Technology: Methods and Approaches. London: Rowman & Littlefield. Pp. 175–191. Carlsen, H., Dreborg, K.H., Godman, M., Hansson, S.O., Johansson, L., & Wikman-Svahn, P. 2010. Assessing socially disruptive technological change. Technology in Society 32: 209–218. Christensen, C.M. 1997. The innovator's dilemma: when new technologies cause great firms to fail. Boston, MA: Harvard Business School Press. Ellsberg, D. 1961. Risk, ambiguity, and the Savage axioms. Quarterly Journal of Economics 75: 643–669. Feinberg, J. 1984. The Moral Limits of the Criminal Law, Volume One: Harm to Others. New York: Oxford University Press. Fox, R. 2000. Medical Uncertainty Revisited. In G.L. Albrecht, R. Fitzpatrick, & S.C. Scrimshaw, eds., Handbook of Social Studies in Health and Medicine. Sage. Pp. 409– 425. Fox, R. & Swazey, J. 2002. Spare Parts: Organ Replacement in American Society (2nd Ed.). New Brunswick, NJ: Transaction Publishers. 14 Freeston, M.H., Rheaume, J., Letart, H., Dugas, M.J., & Ladouceur, R. 1994. Why do people worry? Personality and Individual Differences 17: 791–802. Hamilton, D. 2012. A History of Organ Transplantation: Ancient Legends to Modern Practice. University of Pittsburgh Press. Hansson, S.O. 2013. The Ethics of Risk. Palgrave Macmillan. Harris, J. & Erin, C. 2002. An ethically defensible market in organs: a single buyer like the NHS is an answer. BMJ 325: 114–115. Iliadis, A., & Russo, F. 2016. Critical data studies: an introduction. Big Data & Society. July– Dec.:1–7 Lazar, S. 2018. In dubious battle: uncertainty and the ethics of killing. Philosophical Studies 175: 859–883. Lockhart, T. 2000. Moral Uncertainty and Its Consequences. New York: Oxford University Press. Macaskill, W. 2016. Normative uncertainty as a voting problem. Mind 125: 967–1004. Madden, M., Gilman, M., Levy, K., & Marwick, A. 2017. Privacy, poverty, and big data: a matrix of vulnerabilities for poor Americans. Washington University Law Review 95, 1: 53–125. Markides, C. 2006. Disruptive innovation: in need of better theory. The Journal of Product Innovation Management 23: 19–25. Mittelstadt, B.D., & Floridi, L. 2015. The ethics of big data: current and foreseeable issues in biomedical contexts. Science and Engineering Ethics. DOI 10.1007/s11948-015-96522 Nagel, T. 1993. Moral luck. In D. Statman, ed., Moral Luck. State University of New York Press, Albany, New York. Pp. 57–71. Nickel, P.J. 2019. The ethics of uncertainty for data subjects. In J. Krutzinna & L. Floridi, eds., The Ethics of Medical Data Donation. Springer. Pp. 55–74. Nyholm, S. 2016. The normative and evaluative status of moral distress in health care contexts. American Journal of Bioethics 16: 17–19. Richardson, R. 2006. Human dissection and organ donation: a historical and social background. Mortality 11: 151–165. Rosenberger, R. 2014. Multistability and the agency of mundane artifacts: from speed bumps to subway benches. Human Studies 37: 369–392. Scheper-Hughes, N. 2007. The Tyranny of the Gift: Sacrificial Violence in Living Donor Transplants. American Journal of Transplantation 7, 3: 507–511. Slutsky, A.S. 2015. History of mechanical ventilation: from Vesalius to ventilator-induced lung injury. American Journal of Respiratory and Critical Care Medicine 191, 10: 1106–1115. Walzer, M. 1974. Political Action: The Problem of Many Hands. In M. Cohen, T. Nagel, & T. Scanlon, eds., War and Moral Responsibility. Princeton University Press. Pp. 62–82. Zimmerman, M.J. 2008. Living with Uncertainty: The Moral Significance of Ignorance. Cambridge University Press.