When our choice affects some other person and the outcome is unknown, it has been argued that we should defer to their risk attitude, if known, or else default to use of a risk avoidant risk function. This, in turn, has been claimed to require the use of a risk avoidant risk function when making decisions that primarily affect future people, and to decrease the desirability of efforts to prevent human extinction, owing to the significant (...) risks associated with continued human survival. I raise objections to the claim that respect for others’ risk attitudes requires risk avoidance when choosing for future generations. In particular, I argue that there is no known principle of interpersonal aggregation that yields acceptable results in variable population contexts and is consistent with a plausible ideal of respect for others’ risk attitudes in fixed population cases. GPI Working Paper No . 20-2022. (shrink)

Because of accelerating technological progress, humankind may be rapidly approaching a critical phase in its career. In addition to well-known threats such as nuclear holocaust, the propects of radically transforming technologies like nanotech systems and machine intelligence present us with unprecedented opportunities and risks. Our future, and whether we will have a future at all, may well be determined by how we deal with these challenges. In the case of radically transforming technologies, a better understanding of the transition dynamics from (...) a human to a "posthuman" society is needed. Of particular importance is to know where the pitfalls are: the ways in which things could go terminally wrong. While we have had long exposure to various personal, local, and endurable global hazards, this paper analyzes a recently emerging category: that of existential risks. These are threats that could case our extinction or destroy the potential of Earth - originating intelligent life. Some of these threats are relatively well known while others, including some of the gravest, have gone almost unrecognized. Existential risks have a cluster of features that make ordinary risk management ineffective. A final section of this paper discusses several ethical and policy implications. A clearer understanding of the threat picture will enable us to formulate better strategies. (shrink)

In this paper, we argue that our moral concern for future wellbeing should reduce over time due to important practical considerations about how humans interact with spacetime. After surveying several of these considerations (around equality, special duties, existential contingency, and overlapping moral concern) we develop a set of core principles that can both explain their moral significance and highlight why this is inherently bound up with our relationship with spacetime. These relate to the equitable distribution of: 1. Moral concern (...) in a universe that we can only causally affect in one temporal direction; 2. Access to the benefits from using spacetime as a resource; and 3. The burdens of care given 1 and 2 We conclude by considering the practical implications of our argument and find that, while it is often assumed that a preference for present over future wellbeing weakens the case for existential risk mitigation, this likely is not the case. (shrink)

A small but growing number of studies have aimed to understand, assess and reduce existential risks, or risks that threaten the continued existence of mankind. However, most attention has been focused on known and tangible risks. This paper proposes a heuristic for reducing the risk of black swan extinction events. These events are, as the name suggests, stochastic and unforeseen when they happen. Decision theory based on a fixed model of possible outcomes cannot properly deal with this kind (...) of event. Neither can probabilistic risk analysis. This paper will argue that the approach that is referred to as engineering safety could be applied to reducing the risk from black swan extinction events. It will also propose a conceptual sketch of how such a strategy may be implemented: isolated, self-sufficient, and continuously manned underground refuges. Some characteristics of such refuges are also described, in particular the psychosocial aspects. Furthermore, it is argued that this implementation of the engineering safety strategy safety barriers would be effective and plausible and could reduce the risk of an extinction event in a wide range of possible scenarios. Considering the staggering opportunity cost of an existential catastrophe, such strategies ought to be explored more vigorously. (shrink)

Humanity stands at a precipice. -/- Our species could survive for millions of generations — enough time to end disease, poverty, and injustice; to reach new heights of flourishing. But this vast future is at risk. With the advent of nuclear weapons, humanity entered a new age, gaining the power to destroy ourselves, without the wisdom to ensure we won’t. Since then, these dangers have only multiplied, from climate change to engineered pandemics and unaligned artificial intelligence. If we do (...) not act fast to reach a place of safety, it may soon be too late. -/- The Precipice explores the science behind the risks we face. It puts them in the context of the greater story of humanity: showing how ending these risks is among the most pressing moral issues of our time. And it points the way forward, to the actions and strategies we can take today to safeguard humanity’s future. (shrink)

We outline an argument favoring voluntary moral bioenhancement as a response to existential risks humanity exposes itself to. We consider this type of enhancement a solution to the antithesis between the extinction of humanity and the imperative of humanity to survive at any cost. By opting for voluntary moral bioenhancement; we refrain from advocating illiberal or even totalitarian strategies that would allegedly help humanity preserve itself. We argue that such strategies; by encroaching upon the freedom of individuals; already inflict (...) a degree of existential harm on human beings. We also give some pointers as to the desirable direction for morally enhanced post-personhood. (shrink)

Human civilisation faces a range of existential risks, including nuclear war, runaway climate change and superintelligent artificial intelligence run amok. As we show here with calculations for the New Zealand setting, large numbers of currently living and, especially, future people are potentially threatened by existential risks. A just process for resource allocation demands that we consider future generations but also account for solidarity with the present. Here we consider the various ethical and policy issues involved and make a (...) case for further engagement with the New Zealand public to determine societal values towards future lives and their protection. (shrink)

Because of accelerating technological progress, humankind may be rapidly approaching a critical phase in its career. In addition to well-known threats such as nuclear holocaust, the prospects of radically transforming technologies like nanotech systems and machine intelligence present us with unprecedented opportunities and risks. Our future, and whether we will have a future at all, may well be determined by how we deal with these challenges. In the case of radically transforming technologies, a better understanding of the transition dynamics from (...) a human to a “posthuman” society is needed. Of particular importance is to know where the pitfalls are: the ways in which things could go terminally wrong. While we have had long exposure to various personal, local, and endurable global hazards, this paper analyzes a recently emerging category: that of existential risks. These are threats that could cause our extinction or destroy the potential of Earth-originating intelligent life. Some of these threats are relatively well known while others, including some of the gravest, have gone almost unrecognized. Existential risks have a cluster of features that make ordinary risk management ineffective. A final section of this paper discusses several ethical and policy implications. A clearer understanding of the threat picture will enable us to formulate better strategies. (shrink)

This paper examines and analyzes five definitions of ‘existential risk.’ It tentatively adopts a pluralistic approach according to which the definition that scholars employ should depend upon the particular context of use. More specifically, the notion that existential risks are ‘risks of human extinction or civilizational collapse’ is best when communicating with the public, whereas equating existential risks with a ‘significant loss of expected value’ may be the most effective definition for establishing existential risk (...) studies as a legitimate field of scientific and philosophical inquiry. In making these arguments, the present paper hopes to provide a modicum of clarity to foundational issues relating to the central concept of arguably the most important discussion of our times. (shrink)

The standard argument to the conclusion that artificial intelligence (AI) constitutes an existential risk for the human species uses two premises: (1) AI may reach superintelligent levels, at which point we humans lose control (the ‘singularity claim’); (2) Any level of intelligence can go along with any goal (the ‘orthogonality thesis’). We find that the singularity claim requires a notion of ‘general intelligence’, while the orthogonality thesis requires a notion of ‘instrumental intelligence’. If this interpretation is correct, they (...) cannot be joined as premises and the argument for the existential risk of AI turns out invalid. If the interpretation is incorrect and both premises use the same notion of intelligence, then at least one of the premises is false and the orthogonality thesis remains itself orthogonal to the argument to existential risk from AI. In either case, the standard argument for existential risk from AI is not sound.—Having said that, there remains a risk of instrumental AI to cause very significant damage if designed or used badly, though this is not due to superintelligence or a singularity. (shrink)

Large-scale, self-sufficient space colonization is a plausible means of efficiently reducing existential risks and ensuring our long-term survival. But humanity is by and large myopic, and as an intergenerational global public good, existential risk reduction is systematically undervalued, hampered by intergenerational discounting. This paper explores how these issues apply to space colonization, arguing that the motivational and psychological barriers to space colonization are a special—and especially strong—case of a more general problem. The upshot is not that large-scale, (...) self-sufficient space colonization will never occur, but that, absent institutional change, the conditions under which it is most likely to occur are precisely those conditions where the threat of suffering risks might be most high. (shrink)

Ian Stoner has recently argued that we ought not to colonize Mars because doing so would flout our pro tanto obligation not to violate the principle of scientific conservation, and there is no countervailing considerations that render our violation of the principle permissible. While I remain agnostic on, my primary goal in this article is to challenge : there are countervailing considerations that render our violation of the principle permissible. As such, Stoner has failed to establish that we ought not (...) to colonize Mars. I close with some thoughts on what it would take to show that we do have an obligation to colonize Mars and related issues concerning the relationship between the way we discount our preferences over time and projects with long time horizons, like space colonization. (shrink)

In this article, I present a critique of Robert Geraci's Apocalyptic artificial intelligence (AI) discourse, drawing attention to certain shortcomings which become apparent when the analytical lens shifts from religion to the race–religion nexus. Building on earlier work, I explore the phenomenon of existential risk associated with Apocalyptic AI in relation to “White Crisis,” a modern racial phenomenon with premodern religious origins. Adopting a critical race theoretical and decolonial perspective, I argue that all three phenomena are entangled and (...) they should be understood as a strategy, albeit perhaps merely rhetorical, for maintaining white hegemony under nonwhite contestation. I further suggest that this claim can be shown to be supported by the disclosure of continuity through change in the long‐durée entanglement of race and religion associated with the establishment, maintenance, expansion, and refinement of the modern/colonial world system if and when such phenomena are understood as iterative shifts in a programmatic trajectory of domination which might usefully be framed as “algorithmic racism.”. (shrink)

Longtermists have recently argued that it is overwhelmingly important to do what we can to mitigate existential risks to humanity. I consider three mistakes that are often made in calculating the value of existential risk mitigation: focusing on cumulative risk rather than period risk; ignoring background risk; and neglecting population dynamics. I show how correcting these mistakes pushes the value of existential risk mitigation substantially below leading estimates, potentially low enough to threaten (...) the normative case for existential risk mitigation. I use this discussion to draw four positive lessons for the study of existential risk: the importance of treating existential risk as an intergenerational coordination problem; a surprising dialectical flip in the relevance of background risk levels to the case for existential risk mitigation; renewed importance of population dynamics, including the dynamics of digital minds; and a novel form of the cluelessness challenge to longtermism. -/- . (shrink)

Alan Turing, one of the fathers of computing, warned that Artificial Intelligence (AI) could one day pose an existential risk to humanity. Today, recent advancements in the field AI have been accompanied by a renewed set of existential warnings. But what exactly constitutes an existential risk? And how exactly does AI pose such a threat? In this chapter we aim to answer these questions. In particular, we will critically explore three commonly cited reasons for thinking (...) that AI poses an existential threat to humanity: the control problem, the possibility of global disruption from an AI race dynamic, and the weaponization of AI. (shrink)

The possibility of social and technological collapse has been the focus of science fiction tropes for decades, but more recent focus has been on specific sources of existential and global catastrophic risk. Because these scenarios are simple to understand and envision, they receive more attention than risks due to complex interplay of failures, or risks that cannot be clearly specified. In this paper, we discuss the possibility that complexity of a certain type leads to fragility which can function (...) as a source of catastrophic or even existential risk. The paper first reviews a hypothesis by Bostrom about inevitable technological risks, named the vulnerable world hypothesis. This paper next hypothesizes that fragility may not only be a possible risk, but could be inevitable,and would therefore be a subclass or example of Bostrom’s vulnerable worlds. After introducing the titular fragile world hypothesis, the paper details the conditions under which it would be correct, and presents arguments for why the conditions may in fact may apply. Finally, the assumptions and potential mitigations of the new hypothesis are contrasted with those Bostrom suggests. (shrink)

Recenze knihy:Toby Ord, The Precipice. Existential Risk and the Future of Humanity. Bloomsbury Publishing, 2020, 480 s.

A new book by Phil Torres, Morality, Foresight, and Human Flourishing: An Introduction to Existential Risks, is reviewed. Morality, Foresight and Human Flourishing is a primer intended to introduce students and interested scholars to the concepts and literature on existential risk. The book’s core methodology is to outline the various existential risks currently discussed in different disciplines and provides novel strategies for risk mitigation. The book is stylistically engaging, lucid and academically current, providing both novice (...) readers and seasoned scholars with an easy-to-read introduction to risk studies. The book is by far the most engaging and comprehensive volume on risk studies aimed at captivating new scholars to the field. (shrink)

Philosophy &Public Affairs, Volume 51, Issue 4, Page 373-412, Fall 2023.

This paper examines and evaluates a range of methodologies that have been proposed for making useful claims about the probability of phenomena that would contribute to existential risk. Section One provides a brief discussion of the nature of such claims, the contexts in which they tend to be made and the kinds of probability that they can contain. Section Two provides an overview of the methodologies that have been developed to arrive at these probabilities and assesses their advantages (...) and disadvantages. Section Three contains four suggestions to improve best practice in existential risk assessment. These suggestions centre on the types of probabilities used in risk assessment, the role of methodology rankings including the ranking of probabilistic information, the extended use of expert elicitation, and the use of confidence measures to better communicate uncertainty in probability assessments. Finally, Section Four provides an annotated literature review of catastrophic and existential risk probability claims as well as the methodologies that were used to produce each of them. (shrink)

This paper examines and evaluates a range of methodologies that have been proposed for making useful claims about the probability of phenomena that would contribute to existential risk. Section One provides a brief discussion of the nature of such claims, the contexts in which they tend to be made and the kinds of probability that they can contain. Section Two provides an overview of the methodologies that have been developed to arrive at these probabilities and assesses their advantages (...) and disadvantages. Section Three contains four suggestions to improve best practice in existential risk assessment. These suggestions centre on the types of probabilities used in risk assessment, the role of methodology rankings including the ranking of probabilistic information, the extended use of expert elicitation, and the use of confidence measures to better communicate uncertainty in probability assessments. Finally, Section Four provides an annotated literature review of catastrophic and existential risk probability claims as well as the methodologies that were used to produce each of them. (shrink)

Bioethics, Volume 36, Issue 9, Page 1001-1002, November 2022.

Journal of Applied Philosophy, EarlyView.

This paper examines and evaluates a range of methodologies that have been proposed for making useful claims about the probability of phenomena that would contribute to existential risk. Section One provides a brief discussion of the nature of such claims, the contexts in which they tend to be made and the kinds of probability that they can contain. Section Two provides an overview of the methodologies that have been developed to arrive at these probabilities and assesses their advantages (...) and disadvantages. Section Three contains four suggestions to improve best practice in existential risk assessment. These suggestions centre on the types of probabilities used in risk assessment, the role of methodology rankings including the ranking of probabilistic information, the extended use of expert elicitation, and the use of confidence measures to better communicate uncertainty in probability assessments. Finally, Section Four provides an annotated literature review of catastrophic and existential risk probability claims as well as the methodologies that were used to produce each of them. (shrink)

So-called ‘existential risks’ present virtually unlimited reasons for probing them and responses to them further. The ensuing normative pull to respond to such risks thus seems to present us with r...

This paper provides a critique of Bostrom’s concern with existential risks, a critique which relies on Adorno and Horkheimer’s interpretation of the Enlightenment. Their interpretation is used to elicit the inner contradictions of transhumanist thought and to show the invalid premises on which it is based. By first outlining Bostrom’s position this paper argues that transhumanism reverts to myth in its attempt to surpass the human condition. Bostrom’s argument is based on three pillars, Maxipok, Parfitian population ethics and a (...) universal notion of general human values. By attempting to transcend the human condition, to achieve post-humanity, transhumanism reverts to myth. Thus, the aim of this paper is to provide a critical examination of transhumanism which elicits its tacit contradictions. It will also be argued that transhumanism’s focus on a universal, all-encompassing, notion of humanity neglects any concern with actual lived lives. This absence is problematic because it clearly shows that there is a discrepancy, between transhumanism’s claimed concern for all of humanity and the practical implications of proposing a universal notion of humanity. This paper will conclude, that transhumanism’s lack of concern with actual lives is due to its universal and totalising gestures. Gestures which allow for universal claims such as general values or Earth-originating intelligent life. (shrink)

ExcerptRecent advances in the functional power of artificial intelligence (AI) have prompted an urgent warning from industry leaders and researchers concerning its “profound risks to society and humanity.”1 Their open letter is admirable not only for its succinct identification of said risks, which include the mass dissemination of misinformation, loss of jobs, and even the possible extinction of our species, but also for its clear normative framing of the problem: “Should we let machines flood our information channels with propaganda and (...) untruth? Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization?”2. (shrink)

Recent advances in natural language processing have given rise to a new kind of AI architecture: the language agent. By repeatedly calling an LLM to perform a variety of cognitive tasks, language agents are able to function autonomously to pursue goals specified in natural language and stored in a human-readable format. Because of their architecture, language agents exhibit behavior that is predictable according to the laws of folk psychology: they function as though they have desires and beliefs, and then make (...) and update plans to pursue their desires given their beliefs. We argue that the rise of language agents significantly reduces the probability of an existential catastrophe due to loss of control over an AGI. This is because the probability of such an existential catastrophe is proportional to the difficulty of aligning AGI systems, and language agents significantly reduce that difficulty. In particular, language agents help to resolve three important issues related to aligning AIs: reward misspecification, goal misgeneralization, and uninterpretability. (shrink)

Increasing rates of psychiatric problems like depression and anxiety among Swedish youth, predominantly among females, are considered a serious public mental health concern. Multiple studies confirm that psychological as well as existential vulnerability manifest in different ways for youths in Sweden. This multi-method study aimed at assessing existential worldview function by three factors: 1) existential worldview, 2) ontological security, and 3) self-concept, attempting to identify possible protective and risk factors for mental ill-health among female youths at (...) risk for depression and anxiety. The sample comprised ten females on the waiting list at an outpatient psychotherapy clinic for teens and young adults. Results indicated that both functional and dysfunctional factors related to mental health were present, where the quality and availability of significant interpersonal relations seemed to have an important influence. Examples of both an impaired worldview function and a lack of an operating existential worldview were found. Psychotherapeutic implications are discussed. (shrink)

Background and objectives: Physicians are exposed to matters of existential character at work, but little is known about the personal impact of such issues. Methods: To explore how physicians experience and cope with existential aspects of their clinical work and how such experiences affect their professional identities, a qualitative study using individual semistructured interviews has analysed accounts of their experiences related to coping with such challenges. Analysis was by systematic text condensation. The purposeful sample comprised 10 physicians (including (...) three women), aged 33–66 years, residents or specialists in cardiology or cardiothoracic surgery, working in a university hospital with 24-hour emergency service and one general practitioner. Results: Participants described a process by which they were able to develop a capacity for coping with the existential challenges at work. After episodes perceived as shocking or horrible earlier in their career, they at present said that they could deal with death and mostly keep it at a distance. Vulnerability was closely linked to professional responsibility and identity, perceived as a burden to be handled. These demands were balanced by an experience of meaning related to their job, connected to making a difference in their patients’ lives. Belonging to a community of their fellows was a presupposition for coping with the loneliness and powerlessness related to their vulnerable professional position. Conclusions: Physicians’ vulnerability facing life and death has been underestimated. Belonging to caring communities may assist growth and coping on exposure to existential aspects of clinical work and developing a professional identity. (shrink)

Background and objectives: Physicians are exposed to matters of existential character at work, but little is known about the personal impact of such issues.Methods: To explore how physicians experience and cope with existential aspects of their clinical work and how such experiences affect their professional identities, a qualitative study using individual semistructured interviews has analysed accounts of their experiences related to coping with such challenges. Analysis was by systematic text condensation. The purposeful sample comprised 10 physicians , aged (...) 33–66 years, residents or specialists in cardiology or cardiothoracic surgery, working in a university hospital with 24-hour emergency service and one general practitioner.Results: Participants described a process by which they were able to develop a capacity for coping with the existential challenges at work. After episodes perceived as shocking or horrible earlier in their career, they at present said that they could deal with death and mostly keep it at a distance. Vulnerability was closely linked to professional responsibility and identity, perceived as a burden to be handled. These demands were balanced by an experience of meaning related to their job, connected to making a difference in their patients’ lives. Belonging to a community of their fellows was a presupposition for coping with the loneliness and powerlessness related to their vulnerable professional position.Conclusions: Physicians’ vulnerability facing life and death has been underestimated. Belonging to caring communities may assist growth and coping on exposure to existential aspects of clinical work and developing a professional identity. (shrink)

Background In a previous paper in Philos Ethics Humanit Med, the 1937 Swedish novel Sömnlös (Swedish for sleepless) by Vilhelm Moberg was used as background for a thought experiment, in which last century’s progresses concerning the safety of sleeping pills were projected into the future. This gave rise to a theoretical discussion about broad medico-philosophical questions such as (among other things) the concept of pharmaceuticalisation. Methods In this follow-up paper, the theme of insomnia in Sömnlös is complemented by a discussion (...) of the concept of nostalgia. The core of the paper is a theoretical discussion about the benefits and risks of nostalgia, bringing together some aspects of recent psychological research about the construct of nostalgia with the main story line of the novel. Results and Conclusion Nostalgia is portrayed as being, in some sense at least, ultimately beneficial for the protagonist of Sömnlös. This is congruent with recent psychological research. However, the story also shows that nostalgia may lead to problematic behaviours, at least when viewed from a virtue ethics perspective. Hence, nostalgia is both what leads the protagonist into ethically problematic behaviour and that which (paradoxically) ultimately saves him from his initial lack of courage, justice, temperance and practical wisdom. Moreover, the protagonist does not only “grow” ethically but also existentially. Hence, the novel opens up the possibility that insomnia and nostalgia might be viewed as bearers of important existential information (cf. sociologist of religion Peter L. Berger and his concept of “signals of transcendence”). (shrink)

Papers from the conference on AI Risk (published in JETAI), supplemented by additional work. --- If the intelligence of artificial systems were to surpass that of humans, humanity would face significant risks. The time has come to consider these issues, and this consideration must include progress in artificial intelligence (AI) as much as insights from AI theory. -- Featuring contributions from leading experts and thinkers in artificial intelligence, Risks of Artificial Intelligence is the first volume of collected chapters dedicated (...) to examining the risks of AI. The book evaluates predictions of the future of AI, proposes ways to ensure that AI systems will be beneficial to humans, and then critically evaluates such proposals. 1 Vincent C. Müller, Editorial: Risks of Artificial Intelligence - 2 Steve Omohundro, Autonomous Technology and the Greater Human Good - 3 Stuart Armstrong, Kaj Sotala and Sean O’Heigeartaigh, The Errors, Insights and Lessons of Famous AI Predictions - and What they Mean for the Future - 4 Ted Goertzel, The Path to More General Artificial Intelligence - 5 Miles Brundage, Limitations and Risks of Machine Ethics - 6 Roman Yampolskiy, Utility Function Security in Artificially Intelligent Agents - 7 Ben Goertzel, GOLEM: Toward an AGI Meta-Architecture Enabling Both Goal Preservation and Radical Self-Improvement - 8 Alexey Potapov and Sergey Rodionov, Universal Empathy and Ethical Bias for Artificial General Intelligence - 9 András Kornai, Bounding the Impact of AGI - 10 Anders Sandberg, Ethics and Impact of Brain Emulations 11 Daniel Dewey, Long-Term Strategies for Ending Existential Risk from Fast Takeoff - 12 Mark Bishop, The Singularity, or How I Learned to Stop Worrying and Love AI -. (shrink)

The coronavirus pandemic, like its predecessors - AIDS, Ebola, etc., is evidence of the evolutionary instability of the socio-cultural and ecological niche created by mankind, as the main factor in the evolutionary success of our biological species and the civilization created by it. At least, this applies to the modern global civilization, which is called technogenic or technological, although it exists in several varieties. As we hope to show, the current crisis has less ontological as well as epistemological roots; its (...) reason lies in the main evolutionary trends in the development of science as a social institution. It was only later that epistemological factors were transformed into existential-ontological factors associated with the asymmetry of the existence of civilization and our biosocial nature. The perception or ignorance of a risk factor as a real fact is determined by the presence or absence of knowledge about it. In other words, risk is the result of the integration of the corresponding ontological concept into the general categorical structure. The plurality of such structures is a hallmark of multidisciplinary ontologies, each of which is associated with its own factual continuum. The aim of this study was to conceptually model for support relativistic parameter of evolutionary efficiency of stable evolutionary human strategy (SESH) in its techno-rationalistic module. The meaning of this term is equivalent to the category of scientific and technological development. (shrink)

The greatest existential threats to humanity stem from increasingly powerful advanced technologies. Yet the “risk potential” of such tools can only be realized when coupled with a suitable agent who; through error or terror; could use the tool to bring about an existential catastrophe. While the existential risk literature has provided many accounts of how advanced technologies might be misused and abused to cause unprecedented harm; no scholar has yet explored the other half of the (...) agent-tool coupling; namely the agent. This paper aims to correct this failure by offering a comprehensive overview of what we could call “agential riskology.” Only by studying the unique properties of different agential risk types can one acquire an accurate picture of the existential danger before us. (shrink)

Nuclear weapons are different from every other type of weapons technology. Their awesome destructive potential and the unparalleled consequences of their use oblige us to think critically about the ethics of nuclear possession, planning, and use. Joe Nye has given the ethics of nuclear weapons deep consideration. He posits that we have a basic moral obligation to future generations to preserve roughly equal access to important values, including equal chances of survival, and proposes criteria for achieving conditional or “just deterrence” (...) to minimize the risk of nuclear use and help preserve these values. While Nye's conditions are laudable, they are not achievable. They rely on flawed assumptions about the nature of nuclear weapons and the inherent risks of the nuclear deterrence system. Since the Cold War ended, the strategy and practice of nuclear deterrence has grown riskier, more urgent, more dangerous, and less stable. It is time to rethink how we manage nuclear risks. A new nuclear security system must be built on the design principle that the consequences of system failure cannot threaten to end or fundamentally disrupt civilization. We owe the future a new nuclear security strategy that can prevent an existential global nuclear event. (shrink)

If the intelligence of artificial systems were to surpass that of humans significantly, this would constitute a significant risk for humanity. Time has come to consider these issues, and this consideration must include progress in AI as much as insights from the theory of AI. The papers in this volume try to make cautious headway in setting the problem, evaluating predictions on the future of AI, proposing ways to ensure that AI systems will be beneficial to humans – and (...) critically evaluating such proposals. (shrink)

When our choice affects some other person and the outcome is unknown, it has been argued that we should defer to their risk attitude, if known, or else default to use of a risk avoidant risk function. This, in turn, has been claimed to require the use of a risk avoidant risk function when making decisions that primarily affect future people, and to decrease the desirability of efforts to prevent human extinction, owing to the significant (...) risks associated with continued human survival. I raise objections to the claim that respect for others’ risk attitudes requires risk avoidance when choosing for future generations. In particular, I argue that there is no known principle of interpersonal aggregation that yields acceptable results in variable population contexts and is consistent with a plausible ideal of respect for others’ risk attitudes in fixed population cases. GPI Working Paper No. 20-2022. (shrink)

Digitalization of socio-cultural phenomena, including the education system, generates transformations of their qualitative characteristics and parameters, which requires research from the standpoint of methodological analysis and assessment of their possible consequences on humans and society. A significant element of the digital environment, in general, and educational, in particular, is the dialogue, the role of which has both cognitive and ideological, existential, social aspects. The purpose of the research is a philosophical analysis of the digital transformation of dialogue in the (...) context of the educational process, an assessment of its potential and impact on the individual. The research methodology has a complex, interdisciplinary character and is associated with the application of an existential approach to understanding the essence of dialogue, the theory of the development of individual thinking by L.S. Vygotsky, developed by the author of the theories of electronic culture and based on modern developments in the field of the introduction of digital tools in education. The study shows that dialogue in the digital learning environment becomes not so much a form of rapprochement, integration of subjects, as a form of autonomy, isolation, individuals, which changes the nature of learning and has social and existential consequences. This creates the need to transform the digital learning environment when using it from the position of supplementing it with ethical, socio-psychological and emotional components and support tools for the student. Based on the existential-axiological analysis, the positive and risky features of digital dialogue as one of the phenomena of electronic culture and online learning tools are revealed. The concept of digital dialogue in learning, the characteristics of its features, functions, role and influence on the learner in the digital environment are presented. The difficulties of dialogic learning characteristic of the digital environment are revealed, recommendations of a methodological and methodical nature for application in the education system are presented. (shrink)

This article assesses how autonomy and machine learning impact the existential risk of nuclear war. It situates the problem of cyber security, which proceeds by stealth, within the larger context of nuclear deterrence, which is effective when it functions with transparency and credibility. Cyber vulnerabilities poses new weaknesses to the strategic stability provided by nuclear deterrence. This article offers best practices for the use of computer and information technologies integrated into nuclear weapons systems. Focusing on nuclear command and (...) control, avoiding autonomy and machine learning is recommended as one means to reduce the existential risk of unintended nuclear conflict. (shrink)

Sources of evolutionary risk for stable strategy of adaptive Homo sapiens are an imbalance of: (1) the intra-genomic co-evolution (intragenomic conflicts); (2) the gene-cultural co-evolution; (3) inter-cultural co-evolution; (4) techno-humanitarian balance; (5) inter-technological conflicts (technological traps). At least phenomenologically the components of the evolutionary risk are reversible, but in the aggregate they are in potentio irreversible destructive ones for biosocial, and cultural self-identity of Homo sapiens. When the actual evolution is the subject of a rationalist control and/or manipulation, (...) the magnitude of the 4th and 5th components of the evolutionary risk reaches a level of existential significance. (shrink)

The Risk of Being attempts to forge a new language and a new way of reasoning about what it is like to be good and bad by focusing on existential phenomena that reveal what it means to be good and bad.