A classification of the global catastrophic risks of AI is presented, along with a comprehensive list of previously identified risks. This classification allows the identification of several new risks. We show that at each level of AI’s intelligence power, separate types of possible catastrophes dominate. Our classification demonstrates that the field of AI risks is diverse, and includes many scenarios beyond the commonly discussed cases of a paperclip maximizer or robot-caused unemployment. Global catastrophic failure could happen at various levels of (...) AI development, namely, before it starts self-improvement, during its takeoff, when it uses various instruments to escape its initial confinement, or after it successfully takes over the world and starts to implement its goal system, which could be plainly unaligned, or feature-flawed friendliness. AI could also halt at later stages of its development either due to technical glitches or ontological problems. Overall, we identified around several dozen scenarios of AI-driven global catastrophe. The extent of this list illustrates that there is no one simple solution to the problem of AI safety, and that AI safety theory is complex and must be customized for each AI development level. (shrink)

In recent years, anthropic reasoning has been used to justify a number of controversial skeptical hypotheses. In this paper, we consider two prominent examples, viz. Bostrom’s ‘Simulation Argument’ and the problem of ‘Boltzmann Brains’ in big bang cosmology. We argue that these cases call into question the assumption, central to Bayesian confirmation theory, that the relation of evidential confirmation is universally symmetric. We go on to argue that the fact that these arguments appear to contradict this fundamental assumption should not (...) be taken as an immediate refutation, but should rather be seen as indicative of the peculiar role that the relevant hypotheses play in their respective epistemic frameworks. (shrink)

The current assumptions of knowledge acquisition brought about the crisis in the reproducibility of experiments. A complementary perspective should account for the specific causality characteristic of life by integrating past, present, and future. A “second Cartesian revolution,” informed by and in awareness of anticipatory processes, should result in scientific methods that transcend the theology of determinism and reductionism. In our days, science, itself an expression of anticipatory activity, makes possible alternative understandings of reality and its dynamics. For this purpose, the (...) study advances G-complexity for defining and comparing decidable and undecidable knowledge. AI and related computational expressions of knowledge could benefit from the awareness of what distinguishes the dynamics of life from any other expressions of change. (shrink)

A critical survey of some attempts to define ‘computer’, beginning with some informal ones, then critically evaluating those of three philosophers, and concluding with an examination of whether the brain and the universe are computers.

A 1% skeptic is someone who has about a 99% credence in non-skeptical realism and about a 1% credence in the disjunction of all radically skeptical scenarios combined. The first half of this essay defends the epistemic rationality of 1% skepticism, appealing to dream skepticism, simulation skepticism, cosmological skepticism, and wildcard skepticism. The second half of the essay explores the practical behavioral consequences of 1% skepticism, arguing that 1% skepticism need not be behaviorally inert.

The enduring progression of artificial intelligence and cybernetics offers an ever-closer possibility of rational and sentient robots. The ethics and morals deriving from this technological prospect have been considered in the philosophy of artificial intelligence, the design of automatons with roboethics and the contemplation of machine ethics through the concept of artificial moral agents. Across these categories, the robotics laws first proposed by Isaac Asimov in the twentieth century remain well-recognised and esteemed due to their specification of preventing human harm, (...) stipulating obedience to humans and incorporating robotic self-protection. However the overwhelming predominance in the study of this field has focussed on human–robot interactions without fully considering the ethical inevitability of future artificial intelligences communicating together and has not addressed the moral nature of robot–robot interactions. A new robotic law is proposed and termed AIonAI or artificial intelligence-on-artificial intelligence. This law tackles the overlooked area where future artificial intelligences will likely interact amongst themselves, potentially leading to exploitation. As such, they would benefit from adopting a universal law of rights to recognise inherent dignity and the inalienable rights of artificial intelligences. Such a consideration can help prevent exploitation and abuse of rational and sentient beings, but would also importantly reflect on our moral code of ethics and the humanity of our civilisation. (shrink)

Speculative fiction, such as science fiction and fantasy, has a unique epistemic value. We examine similarities and differences between speculative fiction and philosophical thought experiments in terms of how they are cognitively processed. They are similar in their reliance on mental prospection, but dissimilar in that fiction is better able to draw in readers (transportation) and elicit emotional responses. By its use of longer, emotionally poignant narratives and seemingly irrelevant details, speculative fiction allows for a better appraisal of the consequences (...) of philosophical ideas than thought experiments. (shrink)

The sciences occasionally generate discoveries that undermine their own assumptions. Two such discoveries are characterized here: the discovery of apophenia by cognitive psychology and the discovery that physical systems cannot be locally bounded within quantum theory. It is shown that such discoveries have a common structure and that this common structure is an instance of Priest’s well-known Inclosure Schema. This demonstrates that science itself is dialetheic: it generates limit paradoxes. How science proceeds despite this fact is briefly discussed, as is (...) the connection between our results and the realism-antirealism debate. We conclude by suggesting a position of epistemic modesty. (shrink)

After rejecting substance dualism, some naturalists embrace patternism. It states that persons are bodies and that bodies are material machines running abstract person programs. Following Aristotle, these person programs are souls. Patternists adopt four-dimensionalist theories of persistence: Bodies are 3D stages of 4D lives. Patternism permits at least six types of life after death. It permits quantum immortality, teleportation, salvation through advanced technology, promotion out of a simulated reality, computational monadology, and the revision theory of resurrection.

Nick Bostrom’s ‘Simulation Argument’ purports to show that, unless we are confident that advanced ‘posthuman’ civilizations are either extremely rare or extremely rarely interested in running simulations of their own ancestors, we should assign significant credence to the hypothesis that we are simulated. I argue that Bostrom does not succeed in grounding this constraint on credence. I first show that the Simulation Argument requires a curious form of selective scepticism, for it presupposes that we possess good evidence for claims about (...) the physical limits of computation and yet lack good evidence for claims about our own physical constitution. I then show that two ways of modifying the argument so as to remove the need for this presupposition fail to preserve the original conclusion. Finally, I argue that, while there are unusual circumstances in which Bostrom’s selective scepticism might be reasonable, we do not currently find ourselves in such circumstances. There is no good reason to uphold the selective scepticism the Simulation Argument presupposes. There is thus no good reason to believe its conclusion. (shrink)

In the age of ubiquitous technology, humans are reshaped through each transaction they are involved in. AI-driven networks, online games, and multisensory interactive environments make up alternate realities. Within such alternate worlds, users are reshaped as deterministic agents. Technology’s focus on reducing complexity leads to a human being dependent on prediction-driven machines and behaving like them. Meaning and information are disconnected. Existence is reduced to energy processes. The immense gain in efficiency translates as prosperity. Citizens of advanced economies, hurrying in (...) the rhythm of machine-driven interactions, feel entitled to it. Successful at the price of self-awareness, they no longer know what this means. Happiness and prosperity are not consubstantial. Lack of happiness leads to aggression. This is the image of the world as we see it, no longer looking at each other, eye to eye, but screen to screen. The questions eliminated in the process of transferring responsibility from the individual to machines will inevitably become society’s new focus. When the goal is to get everyone to behave like a machine, the Singularity hypothesis becomes self-fulfilling prophecy. If, in addition to having exhausted natural resources, society does not want to end up making the human mind superfluous, it has to seek a better understanding of what makes anticipation possible. As a definitory characteristic of the living, corresponding to its complexity, anticipation can no longer be taken for granted, while every effort is made to reduce complexity for efficiency’s sake. Awareness of the processes conducive to its expression in successful human action will position human beings as masters of their destiny, not slaves of their own making. Antecapere ergo sum might be the counterclaim to Descartes’ Dubito ergo sum . To resist being perfected into oblivion, that is, unsustainable prosperity, means to reclaim the knowledge corresponding to higher levels of complexity. (shrink)

There is no strong reason to believe that human-level intelligence represents an upper limit of the capacity of artificial intelligence, should it be realized. This poses serious safety issues, since a superintelligent system would have great power to direct the future according to its possibly flawed motivation system. Solving this issue in general has proven to be considerably harder than expected. This paper looks at one particular approach, Oracle AI. An Oracle AI is an AI that does not act in (...) the world except by answering questions. Even this narrow approach presents considerable challenges. In this paper, we analyse and critique various methods of controlling the AI. In general an Oracle AI might be safer than unrestricted AI, but still remains potentially dangerous. (shrink)

This paper discusses the relation between intelligence and motivation in artificial agents, developing and briefly arguing for two theses. The first, the orthogonality thesis, holds (with some caveats) that intelligence and final goals (purposes) are orthogonal axes along which possible artificial intellects can freely vary—more or less any level of intelligence could be combined with more or less any final goal. The second, the instrumental convergence thesis, holds that as long as they possess a sufficient level of intelligence, agents having (...) any of a wide range of final goals will pursue similar intermediary goals because they have instrumental reasons to do so. In combination, the two theses help us understand the possible range of behavior of superintelligent agents, and they point to some potential dangers in building such an agent. (shrink)

The Simulation Argument and the Doomsday Argument share certain structural similarities, and hence are often discussed together. Both are cases where reflecting on one’s location among a set of possibilities yields a counter-intuitive conclusion—in one case that the end of humankind is closer than you initially thought, and in the second case that it is more likely than you initially thought that you are living in a computer simulation. Indeed, the two arguments do share strong structural similarities. But there are (...) also some disanalogies between the two arguments, and I argue that these disanalogies mean that the Simulation Argument succeeds and the Doomsday Argument fails. (shrink)

Around the middle of the twentieth century, psychologists rediscovered the value of make-believe. Beginning in the 1940s and 1950s, there was a sudden and considerable outpouring of books that explored the pedagogical and therapeutic significance of imaginative play. Numerous experimental studies published since then have emphasized the importance of games of make-believe in the cognitive development and successful socialization of the very young.1 And increased attention to the use of mental imagery and fantasy in various forms of psychotherapy over the (...) past half century has led many students of human nature to an increased awareness of what Dorothy and Jerome Singer refer to (in their monumental study The House .. (shrink)

A god is a cosmic designer-creator. Atheism says the number of gods is 0. But it is hard to defeat the minimal thesis that some possible universe is actualized by some possible god. Monotheists say the number of gods is 1. Yet no degree of perfection can be coherently assigned to any unique god. Lewis says the number of gods is at least the second beth number. Yet polytheists cannot defend an arbitrary plural number of gods. An alternative is that, (...) for every ordinal, there is a god whose perfection is proportional to it. The n -th god actualizes the best universe(s) in the n -th level of an axiological hierarchy of possible universes. Despite its unorthodoxy, ordinal polytheism has many metaphysically attractive features and merits more serious study. (shrink)

Bayesian probabilistic explication of inductive inference conflates neutrality of supporting evidence for some hypothesis H (“not supporting H”) with disfavoring evidence (“supporting not-H”). This expressive inadequacy leads to spurious results that are artifacts of a poor choice of inductive logic. I illustrate how such artifacts have arisen in simple inductive inferences in cosmology. In the inductive disjunctive fallacy, neutral support for many possibilities is spuriously converted into strong support for their disjunction. The Bayesian “doomsday argument” is shown to rely entirely (...) on a similar artifact, for the result disappears in a reanalysis that employs fragments of inductive logic able to represent evidential neutrality. Finally, the mere supposition of a multiverse is not yet enough to warrant the introduction of probabilities without some factual analog of a randomizer over the multiverses. (shrink)

The evolution of life on Earth has produced an organism that is beginning to model and understand its own evolution and the possible future evolution of life in the universe. These models and associated evidence show that evolution on Earth has a trajectory. The scale over which living processes are organized cooperatively has increased progressively, as has its evolvability. Recent theoretical advances raise the possibility that this trajectory is itself part of a wider developmental process. According to these theories, the (...) developmental process has been shaped by a yet larger evolutionary dynamic that involves the reproduction of universes. This evolutionary dynamic has tuned the key parameters of the universe to increase the likelihood that life will emerge and produce outcomes that are successful in the larger process (e.g. a key outcome may be to produce life and intelligence that intentionally reproduces the universe and tunes the parameters of ‘offspring’ universes). Theory suggests that when life emerges on a planet, it moves along this trajectory of its own accord. However, at a particular point evolution will continue to advance only if organisms emerge that decide to advance the developmental process intentionally. The organisms must be prepared to make this commitment even though the ultimate nature and destination of the process is uncertain, and may forever remain unknown. Organisms that complete this transition to intentional evolution will drive the further development of life and intelligence in the universe. Humanity’s increasing understanding of the evolution of life in the universe is rapidly bringing it to the threshold of this major evolutionary transition. (shrink)

In this philosophical paper, we explore computational and biological analogies to address the fine-tuning problem in cosmology. We first clarify what it means for physical constants or initial conditions to be fine-tuned. We review important distinctions such as the dimensionless and dimensional physical constants, and the classification of constants proposed by Lévy-Leblond. Then we explore how two great analogies, computational and biological, can give new insights into our problem. This paper includes a preliminary study to examine the two analogies. Importantly, (...) analogies are both useful and fundamental cognitive tools, but can also be misused or misinterpreted. The idea that our universe might be modelled as a computational entity is analysed, and we discuss the distinction between physical laws and initial conditions using algorithmic information theory. Smolin introduced the theory of "cosmological natural selection" with a biological analogy in mind. We examine an extension of. (shrink)

I explore physics implications of the External Reality Hypothesis (ERH) that there exists an external physical reality completely independent of us humans. I argue that with a sufficiently broad definition of mathematics, it implies the Mathematical Universe Hypothesis (MUH) that our physical world is an abstract mathematical structure. I discuss various implications of the ERH and MUH, ranging from standard physics topics like symmetries, irreducible representations, units, free parameters, randomness and initial conditions to broader issues like consciousness, parallel universes and (...) Gödel incompleteness. I hypothesize that only computable and decidable (in Gödel’s sense) structures exist, which alleviates the cosmological measure problem and may help explain why our physical laws appear so simple. I also comment on the intimate relation between mathematical structures, computations, simulations and physical systems. (shrink)

If a brain is duplicated so that there are two brains in identical states, are there then two numerically distinct phenomenal experiences or only one? There are two, I argue, and given computationalism, this has implications for what it is to implement a computation. I then consider what happens when a computation is implemented in a system that either uses unreliable components or possesses varying degrees of parallelism. I show that in some of these cases there can be, in a (...) deep and intriguing sense, a fractional (non-integer) number of qualitatively identical phenomenal experiences. This, in turn, has implications for what lessons one should draw from neural replacement scenarios such as Chalmers. (shrink)

I reply to some recent comments by Brian Weatherson on my 'simulation argument'. I clarify some interpretational matters, and address issues relating to epistemological externalism, the difference from traditional brain-in-a-vat arguments, and a challenge based on 'grue'-like predicates.

The purpose of this paper is to provide an account of the epistemology and metaphysics of universe creation on a computer. The paper begins with F.J.Tipler's argument that our experience is indistinguishable from the experience of someone embedded in a perfect computer simulation of our own universe, hence we cannot know whether or not we are part of such a computer program ourselves. Tipler's argument is treated as a special case of epistemological scepticism, in a similar vein to `brain-in-a-vat' arguments. (...) It is argued that the hypothesis that our universe is a program running on a digital computer in another universe generates empirical predictions, and is therefore a falsifiable hypothesis. The computer program hypothesis is also treated as a hypothesis about what exists beyond the physical world, and is compared with Kant's metaphysics of noumena. It is proposed that a theory about what exists beyond the physical world should be formulated with the precise concepts of mathematics, and should generate physical predictions. It is argued that if our universe is a program running on a digital computer, then our universe must have compact spatial topology, and the possibilities of observationally testing this prediction are considered. The possibility of testing the computer program hypothesis with the value of the density parameter Omega_0 is also analysed. The informational requirements for a computer to represent a universeexactly and completely are considered. Consequent doubt is thrown upon Tipler's claim that if a hierarchy of computer universes exists, we would not be able to know which `level of implementation' our universe exists at. It is then argued that a digital computer simulation of a universe cannot exist as a universe. However, the paper concludes with the acknowledgement that an analog computer simulation can be objectively related to the thing it represents, hence an analog computer simulation of a universe could, in principle, exist as a universe. (shrink)