Longtermists have recently argued that it is overwhelmingly important to do what we can to mitigate existential risks to humanity. I consider three mistakes that are often made in calculating the value of existential risk mitigation. I show how correcting these mistakes pushes the value of existential risk mitigation substantially below leading estimates, potentially low enough to threaten the normative case for existential risk mitigation. I use this discussion to draw four positive lessons for the study of existential risk. -/- (...) . (shrink)

Artificial intelligence (AI) and robotics are digital technologies that will have significant impact on the development of humanity in the near future. They have raised fundamental questions about what we should do with these systems, what the systems themselves should do, what risks they involve, and how we can control these. - After the Introduction to the field (§1), the main themes (§2) of this article are: Ethical issues that arise with AI systems as objects, i.e., tools made and used (...) by humans. This includes issues of privacy (§2.1) and manipulation (§2.2), opacity (§2.3) and bias (§2.4), human-robot interaction (§2.5), employment (§2.6), and the effects of autonomy (§2.7). Then AI systems as subjects, i.e., ethics for the AI systems themselves in machine ethics (§2.8) and artificial moral agency (§2.9). Finally, the problem of a possible future AI superintelligence leading to a “singularity” (§2.10). We close with a remark on the vision of AI (§3). - For each section within these themes, we provide a general explanation of the ethical issues, outline existing positions and arguments, then analyse how these play out with current technologies and finally, what policy consequences may be drawn. (shrink)

The goal of this paper is to describe the mechanism of the public perception of risk of artificial intelligence. For that we apply the social amplification of risk framework to the public perception of artificial intelligence using data collected from Twitter from 2007 to 2018. We analyzed when and how there appeared a significant representation of the association between risk and artificial intelligence in the public awareness of artificial intelligence. A significant finding is that the image of the risk of (...) AI is mostly associated with existential risks that became popular after the fourth quarter of 2014. The source of that was the public positioning of experts who happen to be the real movers of the risk perception of AI so far instead of actual disasters. We analyze here how this kind of risk was amplified, its secondary effects, what are the varieties of risk unrelated to existential risk, and what is the dynamics of the experts in addressing their concerns to the audience of lay people. (shrink)

So-called ‘existential risks’ present virtually unlimited reasons for probing them and responses to them further. The ensuing normative pull to respond to such risks thus seems to present us with r...

This paper aims to open a dialogue between philosophers working in decision theory and operations researchers and engineers working on decision-making under deep uncertainty. Specifically, we assess the recommendation to follow a norm of robust satisficing when making decisions under deep uncertainty in the context of decision analyses that rely on the tools of Robust Decision-Making developed by Robert Lempert and colleagues at RAND. We discuss two challenges for robust satisficing: whether the norm might derive its plausibility from an implicit (...) appeal to probabilistic representations of uncertainty of the kind that deep uncertainty is supposed to preclude; and whether there is adequate justification for adopting a satisficing norm, as opposed to an optimizing norm that is sensitive to considerations of robustness. We discuss decision-theoretic and voting-theoretic motivations for robust satisficing, and use these motivations to select among candidate formulations of the robust satisficing norm. (shrink)

As we learn more about the human brain, novel biotechnological means to modulate human behaviour and emotional dispositions become possible. These technologies could be used to enhance our morality. Moral bioenhancement, an instance of human enhancement, alters a person’s dispositions, emotions or behaviour in order to make that person more moral. I will argue that moral bioenhancement could be carried out in three different ways. The first strategy, well known from science fiction, is behavioural enhancement. The second strategy, favoured by (...) prominent defenders of moral bioenhancement, is emotional enhancement. The third strategy is the enhancement of moral dispositions, such as empathy and inequity aversion. I will argue that we ought to implement a combination of the second and third strategies. Furthermore, I will argue that the usual arguments against other instances of human enhancement do not apply to moral bioenhancement, or apply only to the first strategy, behavioural enhancement. (shrink)

The biosphere represents the global sum of all ecosystems. According to a prominent view in environmental ethics, ecocentrism, these ecosystems matter for their own sake, and not only because they contribute to human ends. As such, some ecocentrists are critical of the modern industrial civilization, and a few even argue that an irreversible collapse of the modern industrial civilization would be a good thing. However, taking a longer view and considering the eventual destruction of the biosphere by astronomical processes, we (...) argue that humans, a species with considerable technological know-how and industrial capacity could intervene to extend the lifespan of Earth’s biosphere, perhaps by several billion years. We argue that human civilization, despite its flaws and harmful impacts on many ecosystems, is the biosphere’s best hope of avoiding premature destruction. We argue that proponents of ecocentrism, even those who wholly disregard anthropocentric values, have a strong moral reason preserve the modern industrial civilization, for as long as needed to ensure biosphere survival. (shrink)

The well-known argument against the Search for ExtraTerrestrial Intelligence (SETI) due to George Gaylord Simpson is re-analyzed almost half a century later, in the light of our improved understanding of preconditions for the emergence of life and intelligence brought about by the ongoing “astrobiological revolution”. Simpson’s argument has been enormously influential, in particular in biological circles, and it arguably fueled the most serious opposition to SETI programmes and their funding. I argue that both proponents and opponents of Simpson’s argument have (...) occasionally mispresented its core content. Proponents often oversimplify it as just another consequence of biological contingency, thus leaving their position open to general arguments limiting the scope of contingency in evolution (such as the recent argument of Geerat Vermeij based on selection effects in the fossil record). They also tend to neglect that the argument has been presented as essentially atemporal, while referring to entities and processes that are likely to change over time; this has become even less justifiable as our astrobiological knowledge increased in recent years. Opponents have failed to see that the weaknesses in Simpson’s position could be removed by restructuring of the argument; I suggest one way of such restructuring, envisioned long ago in the fictional context by Stanislaw Lem. While no firm consensus has emerged on the validity of Simpson’s argument so far, I suggest that, contrary to the original motivation, today it is less an anti-SETI argument, and more an astrobiological research programme. In this research programme, SETI could be generalized into a platform for testing some of the deepest assumptions about evolutionary continuity and the relative role of contingency versus convergence on unprecedented spatial and temporal scales. (shrink)

An advanced artificial intelligence could pose a significant existential risk to humanity. Several research institutes have been set-up to address those risks. And there is an increasing number of academic publications analysing and evaluating their seriousness. Nick Bostrom’s superintelligence: paths, dangers, strategies represents the apotheosis of this trend. In this article, I argue that in defending the credibility of AI risk, Bostrom makes an epistemic move that is analogous to one made by so-called sceptical theists in the debate about the (...) existence of God. And while this analogy is interesting in its own right, what is more interesting are its potential implications. It has been repeatedly argued that sceptical theism has devastating effects on our beliefs and practices. Could it be that AI-doomsaying has similar effects? I argue that it could. Specifically, and somewhat paradoxically, I argue that it could amount to either a reductio of the doomsayers position, or an important and additional reason to join their cause. I use this paradox to suggest that the modal standards for argument in the superintelligence debate need to be addressed. (shrink)

In a recent editorial, Raymond Spier expresses skepticism over claims that climate change is driven by human actions and that humanity should act to avoid climate change. This paper responds to this skepticism as part of a broader review of the science and ethics of climate change. While much remains uncertain about the climate, research indicates that observed temperature increases are human-driven. Although opinions vary regarding what should be done, prominent arguments against action are based on dubious factual and ethical (...) positions. Thus, the skepticisms in the recent editorial are unwarranted. This does not diminish the general merits of skeptical intellectual inquiry. (shrink)

Human civilisation faces a range of existential risks, including nuclear war, runaway climate change and superintelligent artificial intelligence run amok. As we show here with calculations for the New Zealand setting, large numbers of currently living and, especially, future people are potentially threatened by existential risks. A just process for resource allocation demands that we consider future generations but also account for solidarity with the present. Here we consider the various ethical and policy issues involved and make a case for (...) further engagement with the New Zealand public to determine societal values towards future lives and their protection. (shrink)

When our choice affects some other person and the outcome is unknown, it has been argued that we should defer to their risk attitude, if known, or else default to use of a risk avoidant risk function. This, in turn, has been claimed to require the use of a risk avoidant risk function when making decisions that primarily affect future people, and to decrease the desirability of efforts to prevent human extinction, owing to the significant risks associated with continued human (...) survival. I raise objections to the claim that respect for others’ risk attitudes requires risk avoidance when choosing for future generations. In particular, I argue that there is no known principle of interpersonal aggregation that yields acceptable results in variable population contexts and is consistent with a plausible ideal of respect for others’ risk attitudes in fixed population cases. GPI Working Paper No . 20-2022. (shrink)

This paper aims to open a dialogue between philosophers working in decision theory and operations researchers and engineers whose research addresses the topic of decision making under deep uncertainty. Specifically, we assess the recommendation to follow a norm of robust satisficing when making decisions under deep uncertainty in the context of decision analyses that rely on the tools of Robust Decision Making developed by Robert Lempert and colleagues at RAND. We discuss decision-theoretic and voting-theoretic motivations for robust satisficing, then use (...) these motivations to select among candidate formulations of the robust satisficing norm. We also discuss two challenges for robust satisficing: whether the norm might in fact derive its plausibility from an implicit appeal to probabilistic representations of uncertainty of the kind that deep uncertainty is supposed to preclude; and whether there is adequate justification for adopting a satisficing norm, as opposed to an optimizing norm that is sensitive to considerations of robustness. GPI Working Paper No. 15-2020. (shrink)

Scientific and technological progress might change people’s capabilities or incentives in ways that would destabilize civilization. For example, advances in DIY biohacking tools might make it easy for anybody with basic training in biology to kill millions; novel military technologies could trigger arms races in which whoever strikes first has a decisive advantage; or some economically advantageous process may be invented that produces disastrous negative global externalities that are hard to regulate. This paper introduces the concept of a vulnerable world: (...) roughly, one in which there is some level of technological development at which civilization almost certainly gets devastated by default, i.e. unless it has exited the “semi-anarchic default condition”. Several counterfactual historical and speculative future vulnerabilities are analyzed and arranged into a typology. A general ability to stabilize a vulnerable world would require greatly amplified capacities for preventive policing and global governance. The vulnerable world hypothesis thus offers a new perspective from which to evaluate the risk-benefit balance of developments towards ubiquitous surveillance or a unipolar world order. (shrink)

Various kinds of human bioenhancement represent a major topic of contention in both bioethics and futures studies. Moral enhancement is one of them. It will be argued that voluntary moral bio-enhancement should be based on an opt out moral enhancement scheme. Such a scheme would avoid the challenges of a voluntary moral enhancement opt in scheme. The former has a proper place in a minimal state. It will be explained why such a state can be called Utopia. The concept of (...) voluntary opt out moral enhancement in Utopia will be highlighted in detail. (shrink)

A lengthy reply to 13 critical discussions of _Meaning in Life: An Analytic Study_ collected in an e-book and reprinted from the _Journal of Philosophy of Life_. The contributors are from a variety of philosophical traditions, including the Anglo-American, Continental and East Asian (especially Buddhist and Japanese) ones.