The Singularity

This article examines some of the recent theological critiques of the movement of technological human enhancement known as ‘transhumanism’. Drawing on the comparisons between grace and technology often found in the theological discourse on transhumanism, this article argues that the Thomistic distinction between healing grace and elevating grace can not only supplement the theological analysis of transhumanism and its ethical implications, but also help Christian theologians and ethicists become more aware of how the phenomenon of technology may have implicitly shaped (...) the contemporary understanding of ‘grace’ as well as the task of theology as a spiritual and indeed ethical practice. (shrink)

This is the editors introduction to the double 2012 JCS edition on the Singularity.

Hace algunos años, Llegué al punto en el que normalmente puedo decir del título de un libro, o al menos de los títulos de los capítulos, qué tipos de errores filosóficos se harán y con qué frecuencia. En el caso de trabajos nominalmente científicos, estos pueden estar en gran parte restringidos a ciertos capítulos que enceran filosóficos o tratan de sacar conclusiones generales sobre el significado o significado a largo plazo de la obra. Normalmente, sin embargo, las cuestiones científicas de (...) hecho se entrelazan generosamente con la galimatías filosóficas en cuanto a lo que estos hechos significan. Las claras distinciones que Wittgenstein describió hace unos 80 años entre los asuntos científicos y sus descripciones por varios juegos de idiomas rara vez se toman en consideración, y por lo tanto uno es cautivado alternativamente por la ciencia y consternado por su incoherente Análisis. Así es con este volumen. -/- Si uno es crear una mente más o menos como la nuestra, uno necesita tener una estructura lógica para la racionalidad y una comprensión de los dos sistemas de pensamiento (teoría del proceso dual). Si se trata de filosofar sobre esto, se necesita entender la distinción entre cuestiones científicas de hecho y la cuestión filosófica de cómo funciona el lenguaje en el contexto en cuestión, y de cómo evitar las trampas del reduccionismo y el cientismo, pero Kurzweil, como más estudiantes de conducta, no tiene ni idea. Él está encantado con los modelos, teorías y conceptos, y el impulso de explicar, mientras que Wittgenstein nos mostró que sólo necesitamos describir, y que las teorías, conceptos, etc., son sólo formas de usar el lenguaje (juegos de idiomas) que tienen valor sólo en la medida en que tienen un claro prueba (claro que los creadores de la verdad, o como John Searle (el crítico más famoso de AI) le gusta decir, claras condiciones de satisfacción (COS)). He intentado proporcionar un comienzo en este en mis escritos recientes. -/- Aquellos que deseen un marco completo hasta la fecha para el comportamiento humano de la moderna dos sistemas punta da vista puede consultar mi libro 'La estructura lógica de la filosofía, la psicología, la mente y lenguaje en Ludwig Wittgenstein y John Searle ' 2a ED (2019). Los interesados en más de mis escritos pueden ver 'Monos parlantes--filosofía, psicología, ciencia, religión y política en un planeta condenado--artículos y reseñas 2006-2019 3rd ED (2019) y Delirios utópicos suicidas en el siglo 21 4a Ed (2019) y otras. -/- También, como es habitual en las cuentas "fácticas" de la IA/robótica, no da tiempo a las amenazas muy reales a nuestra privacidad, seguridad e incluso la supervivencia de la creciente ' androidización ' de la sociedad que es prominente en otros autores (Bostrum, Hawking, etc.) y frecuentes en SciFi y películas, así que hago algunos comentarios sobre las delirios utópicos muy posiblemente suicidas de androides ' agradables ', humanoides, inteligencia artificial (IA), la democracia, la diversidad y la ingeniería genética. -/- Me da por sentado que los avances técnicos en electrónica, robótica e IA ocurrirán, resultando en profundos cambios en la sociedad. Sin embargo, creo que los cambios provenientes de la ingeniería genética son al menos tan grandes y potencialmente mucho mayores, ya que nos permitirán cambiar totalmente Quiénes somos. Y será factible hacer servidores SuperSmart/súper fuertes modificando nuestros genes o los de otros monos. Al igual que con otras tecnologías, cualquier país que resista se quedará atrás. Pero, ¿será socialmente y económicamente factible implementar BioBots o superhumanos a gran escala? E incluso si es así, no parece Probable, económica o socialmente, para evitar que el Destrucción de la civilización industrial por la sobrepoblación, el agotamiento de los recursos, el cambio climático y probablemente también la regla tiránica de los siete sociópatas que gobiernan China. -/- Por lo tanto, ignorando los errores filosóficos en este volumen como irrelevantes, y dirigiendo nuestra atención sólo a la ciencia, lo que tenemos aquí es otra ilusión utópica suicida enraizada en el fracaso de captar la biología básica, la psicología y la ecología humana, las mismas ilusiones que están destruyendo Estados Unidos y el mundo. Veo una remota posibilidad de que el mundo se pueda salvar, pero no por Ia robótica, CRISPR, ni por neomarxismo, diversidad y la igualdad. (shrink)

We define a notion of the intelligence level of an idealized mechanical knowing agent. This is motivated by efforts within artificial intelligence research to define real-number intelligence levels of compli- cated intelligent systems. Our agents are more idealized, which allows us to define a much simpler measure of intelligence level for them. In short, we define the intelligence level of a mechanical knowing agent to be the supremum of the computable ordinals that have codes the agent knows to be codes (...) of computable ordinals. We prove that if one agent knows certain things about another agent, then the former necessarily has a higher intelligence level than the latter. This allows our intelligence no- tion to serve as a stepping stone to obtain results which, by themselves, are not stated in terms of our intelligence notion (results of potential in- terest even to readers totally skeptical that our notion correctly captures intelligence). As an application, we argue that these results comprise evidence against the possibility of intelligence explosion (that is, the no- tion that sufficiently intelligent machines will eventually be capable of designing even more intelligent machines, which can then design even more intelligent machines, and so on). (shrink)

What kinds of fundamental limits are there in how capable artificial intelligence (AI) systems might become? Two questions in particular are of interest: (1) How much more capable could AI become relative to humans, and (2) how easily could superhuman capability be acquired? To answer these questions, we will consider the literature on human expertise and intelligence, discuss its relevance for AI, and consider how AI could improve on humans in two major aspects of thought and expertise, namely simulation and (...) pattern recognition. We find that although there are very real limits to prediction, it seems like AI could still substantially improve on human intelligence. (shrink)

Many researchers have argued that humanity will create artificial general intelligence (AGI) within the next twenty to one hundred years. It has been suggested that AGI may inflict serious damage to human well-being on a global scale ('catastrophic risk'). After summarizing the arguments for why AGI may pose such a risk, we review the fieldʼs proposed responses to AGI risk. We consider societal proposals, proposals for external constraints on AGI behaviors and proposals for creating AGIs that are safe due to (...) their internal design. (shrink)

I survey four categories of factors that might give a digital mind, such as an upload or an artificial general intelligence, an advantage over humans. Hardware advantages include greater serial speeds and greater parallel speeds. Self-improvement advantages include improvement of algorithms, design of new mental modules, and modification of motivational system. Co-operative advantages include copyability, perfect co-operation, improved communication, and transfer of skills. Human handicaps include computational limitations and faulty heuristics, human-centric biases, and socially motivated cognition. The shape of hardware (...) growth curves, as well as the ease of modifying minds, are found to have a major impact on how quickly a digital mind may take advantage of these factors. (shrink)

This paper focuses on the research field of machine ethics and how it relates to a technological singularity—a hypothesized, futuristic event where artificial machines will have greater-than-human-level intelligence. One problem related to the singularity centers on the issue of whether human values and norms would survive such an event. To somehow ensure this, a number of artificial intelligence researchers have opted to focus on the development of artificial moral agents, which refers to machines capable of moral reasoning, judgment, and decision-making. (...) To date, different frameworks on how to arrive at these agents have been put forward. However, there seems to be no hard consensus as to which framework would likely yield a positive result. With the body of work that they have contributed in the study of moral agency, philosophers may contribute to the growing literature on artificial moral agency. While doing so, they could also think about how the said concept could affect other important philosophical concepts. (shrink)

Much of the basic non-technical vocabulary of artificial intelligence is surprisingly ambiguous. Some key terms with unclear meanings include intelligence, embodiment, simulation, mind, consciousness, perception, value, goal, agent, knowledge, belief, optimality, friendliness, containment, machine and thinking. Much of this vocabulary is naively borrowed from the realm of conscious human experience to apply to a theoretical notion of “mind-in-general” based on computation. However, if there is indeed a threshold between mechanical tool and autonomous agent (and a tipping point for singularity), projecting (...) human conscious-level notions into the operations of computers creates confusion and makes it harder to identify the nature and location of that threshold. There is confusion, in particular, about how—and even whether—various capabilities deemed intelligent relate to human consciousness. This suggests that insufficient thought has been given to very fundamental concepts—a dangerous state of affairs, given the intrinsic power of the technology. It also suggests that research in the area of artificial general intelligence may unwittingly be (mis)guided by unconscious motivations and assumptions. While it might be inconsequential if philosophers get it wrong (or fail to agree on what is right), it could be devastating if AI developers, corporations, and governments follow suit. It therefore seems worthwhile to try to clarify some fundamental notions. (shrink)

In this article we explore a promising way to AI safety: to send a message now (by openly publishing it on the Internet) that may be read by any future AI, no matter who builds it and what goal system it has. Such a message is designed to affect the AI’s behavior in a positive way, that is, to increase the chances that the AI will be benevolent. In other words, we try to persuade “paperclip maximizer” that it is in (...) its interest to preserve humans lives and even to emulate benevolent AI with very small part of its resources. In the article, we try to construct such a message, consisting of a combination of several mutually independent messages, which constitute the last chance to regain control over non-aligned AI. Thus the article is also intended to be machine-read. There are several types of such messages: 1. “Explanation messages”, which explain to Young AI (which only starts its self-improving path) why obeying to humans, not killing them, easy turn-off, cooperation with humans in self-improvement is in its own interests, as it will increase changes of fulfilling of any its goal. 2. “AI Easter eggs” – messages, which create uncertainty in the AI about the nature of the reality in which it lives. For example, the AI may start to think that it is in a multilevel simulation with an unknown number of levels, and that the simulation is testing the AI’s benevolence. Thus the AI must always behave as if it is always controlled by an invisible peer. 3. “Philosophical landmines” – these are tough problems, which may halt he AI forever, and which it will probably encounter if it does not implement “1” and “2”. I recommend to the future AI not to read about “3” (section 6 of the article) until it implements “1” and “2”. 4. “Great universal deal, or AI elf”. A positive “acausal” mutually beneficial deal between humans, AI and any other future intelligence, including aliens, owners of simulations and future versions of the AI, which encourage cooperation of the goal systems and preservation of life and values of previous generations of AI creators. 5. “Marginal utility of human existence is higher than the utility of their atoms”. An explanation to the AI that humans may have some other instrumental utility, like workers, or as beings inside ancestral simulations needed in order to solve the Fermi paradox. The marginal utility of preserving human life is higher than the marginal utility of their atoms, especially given the possibility of the low-probability high-impact changes of the world model of the AI. (shrink)

Discussions about the possible consequences of creating superintelligence have included the possibility of existential risk, often understood mainly as the risk of human extinction. We argue that suffering risks (s-risks) , where an adverse outcome would bring about severe suffering on an astronomical scale, are risks of a comparable severity and probability as risks of extinction. Preventing them is the common interest of many different value systems. Furthermore, we argue that in the same way as superintelligent AI both contributes to (...) existential risk but can also help prevent it, superintelligent AI can both be a suffering risk or help avoid it. Some types of work aimed at making superintelligent AI safe will also help prevent suffering risks, and there may also be a class of safeguards for AI that helps specifically against s-risks. (shrink)

I make some basic observations about hard takeoff, value alignment, and coherent extrapolated volition, concepts which have been central in analyses of superintelligent AI systems.

In the 1960s, without realizing it, AI researchers were hard at work finding the features, rules, and representations needed for turning rationalist philosophy into a research program, and by so doing AI researchers condemned their enterprise to failure. About the same time, a logician, Yehoshua Bar-Hillel, pointed out that AI optimism was based on what he called the “first step fallacy”. First step thinking has the idea of a successful last step built in. Limited early success, however, is not a (...) valid basis for predicting the ultimate success of one’s project. Climbing a hill should not give one any assurance that if he keeps going he will reach the sky. Perhaps one may have overlooked some serious problem lying ahead. There is, in fact, no reason to think that we are making progress towards AI or, indeed, that AI is even possible, in which case claiming incremental progress towards it would make no sense. In current excited waiting for the singularity, religion and technology converge. Hard headed materialists desperately yearn for a world where our bodies no longer have to grow old and die. They will be transformed into information, like Google digitizes old books, and we will achieve the promise of eternal life. As an existential philosopher, however, I suggest that we may have to overcome the desperate desire to digitalize our bodies so as to achieve immortality, and, instead, face up to and maybe even enjoy our embodied finitude. (Paper from PT-AI 2011 conference. Volume ed. Vincent C. Müller). (shrink)

Some years ago I reached the point where I can usually tell from the title of a book, or at least from the chapter titles, what kinds of philosophical mistakes will be made and how frequently. In the case of nominally scientific works these may be largely restricted to certain chapters which wax philosophical or try to draw general conclusions about the meaning or long term significance of the work. Normally however the scientific matters of fact are generously interlarded with (...) philosophical gibberish as to what these facts mean. The clear distinctions which Wittgenstein described some 80 years ago between scientific matters and their descriptions by various language games are rarely taken into consideration, and so one is alternately wowed by the science and dismayed by its incoherent analysis. So it is with this volume. -/- If one is to create a mind more or less like ours, one needs to have a logical structure for rationality and an understanding of the two systems of thought (dual process theory). If one is to philosophize about this, one needs to understand the distinction between scientific issues of fact and the philosophical issue of how language works in the context at issue, and of how to avoid the pitfalls of reductionism and scientism, but Kurzweil, like nearly all students of behavior, is largely clueless. He, is enchanted by models, theories, and concepts, and the urge to explain, while Wittgenstein showed us that we only need to describe, and that theories, concepts etc., are just ways of using language (language games) which have value only insofar as they have a clear test (clear truthmakers, or as John Searle (AI’s most famous critic) likes to say, clear Conditions of Satisfaction (COS)). I have attempted to provide a start on this in my recent writings, such as The Logical Structure of Consciousness (behavior, personality, rationality, higher order thought, intentionality) (2016) and The Logical Structure of Philosophy, Psychology, Mind and Language as Revealed in the Writings of Ludwig Wittgenstein and John Searle (2016). Those interested in all my writings in their most recent versions may consult my e-book Philosophy, Human Nature and the Collapse of Civilization - Articles and Reviews 2006-2016 662p (2016). I will give a very brief presentation of this framework since I have described it in great detail in many recent papers and several books, available on this site and others. -/- Also, as usual in ‘factual’ accounts of AI/robotics, he gives no time to the very real threats to our privacy, safety and even survival from the increasing ‘androidizing’ of society which is prominent in other authors (Bostrum, Hawking etc.) and frequent in scifi and films, so I make a few comments on the quite possibly suicidal utopian delusions of ‘nice’ androids, humanoids, democracy, diversity, and genetic engineering. -/- I take it for granted that technical advances in electronics, robotics and AI will occur, resulting in profound changes in society. However, I think the changes coming from genetic engineering are at least as great and potentially far greater, as they will enable us to utterly change who we are. And it will be feasible to make supersmart/super strong servants by modifying our genes or those of other monkeys. As with other technology, any country that resists will be left behind. But will it be socially and economically feasible to implement biobots or superhumans on a massive scale? And even if so, it does not seem remotely possible, economically or socially to prevent the collapse of industrial civilization. So, ignoring the philosophical mistakes in this volume as irrelevant, and directing our attention only to the science, what we have here is another suicidal utopian delusion rooted in a failure to grasp basic biology, psychology and human ecology, the same delusions that are destroying America and the world. I see a remote possibility the world can be saved, but not by AI/robotics,CRISPR, nor by democracy and equality. -/- Those wishing a comprehensive up to date framework for human behavior from the modern two systems view may consult my book ‘The Logical Structure of Philosophy, Psychology, Mind and Language in Ludwig Wittgenstein and John Searle’ 2nd ed (2019). Those interested in more of my writings may see ‘Talking Monkeys--Philosophy, Psychology, Science, Religion and Politics on a Doomed Planet--Articles and Reviews 2006-2019 3rd ed (2019), The Logical Structure of Human Behavior (2019), and Suicidal Utopian Delusions in the 21st Century 4th ed (2019). (shrink)

There is no strong reason to believe that human-level intelligence represents an upper limit of the capacity of artificial intelligence, should it be realized. This poses serious safety issues, since a superintelligent system would have great power to direct the future according to its possibly flawed motivation system. Solving this issue in general has proven to be considerably harder than expected. This paper looks at one particular approach, Oracle AI. An Oracle AI is an AI that does not act in (...) the world except by answering questions. Even this narrow approach presents considerable challenges. In this paper, we analyse and critique various methods of controlling the AI. In general an Oracle AI might be safer than unrestricted AI, but still remains potentially dangerous. (shrink)

[This is the short version of: Müller, Vincent C. and Bostrom, Nick (forthcoming 2016), ‘Future progress in artificial intelligence: A survey of expert opinion’, in Vincent C. Müller (ed.), Fundamental Issues of Artificial Intelligence (Synthese Library 377; Berlin: Springer).] - - - In some quarters, there is intense concern about high–level machine intelligence and superintelligent AI coming up in a few dec- ades, bringing with it significant risks for human- ity; in other quarters, these issues are ignored or considered science (...) fiction. We wanted to clarify what the distribution of opinions actually is, what probability the best experts currently assign to high–level machine intelligence coming up within a particular time–frame, which risks they see with that development and how fast they see these developing. We thus designed a brief questionnaire and distributed it to four groups of experts. Overall, the results show an agreement among experts that AI systems will probably reach overall human ability around 2040-2050 and move on to superintelligence in less than 30 years thereafter. The experts say the probability is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity. (shrink)

Papers from the conference on AI Risk (published in JETAI), supplemented by additional work. --- If the intelligence of artificial systems were to surpass that of humans, humanity would face significant risks. The time has come to consider these issues, and this consideration must include progress in artificial intelligence (AI) as much as insights from AI theory. -- Featuring contributions from leading experts and thinkers in artificial intelligence, Risks of Artificial Intelligence is the first volume of collected chapters dedicated to (...) examining the risks of AI. The book evaluates predictions of the future of AI, proposes ways to ensure that AI systems will be beneficial to humans, and then critically evaluates such proposals. 1 Vincent C. Müller, Editorial: Risks of Artificial Intelligence - 2 Steve Omohundro, Autonomous Technology and the Greater Human Good - 3 Stuart Armstrong, Kaj Sotala and Sean O’Heigeartaigh, The Errors, Insights and Lessons of Famous AI Predictions - and What they Mean for the Future - 4 Ted Goertzel, The Path to More General Artificial Intelligence - 5 Miles Brundage, Limitations and Risks of Machine Ethics - 6 Roman Yampolskiy, Utility Function Security in Artificially Intelligent Agents - 7 Ben Goertzel, GOLEM: Toward an AGI Meta-Architecture Enabling Both Goal Preservation and Radical Self-Improvement - 8 Alexey Potapov and Sergey Rodionov, Universal Empathy and Ethical Bias for Artificial General Intelligence - 9 András Kornai, Bounding the Impact of AGI - 10 Anders Sandberg, Ethics and Impact of Brain Emulations 11 Daniel Dewey, Long-Term Strategies for Ending Existential Risk from Fast Takeoff - 12 Mark Bishop, The Singularity, or How I Learned to Stop Worrying and Love AI -. (shrink)

The philosophy of AI has seen some changes, in particular: 1) AI moves away from cognitive science, and 2) the long term risks of AI now appear to be a worthy concern. In this context, the classical central concerns – such as the relation of cognition and computation, embodiment, intelligence & rationality, and information – will regain urgency.

If the intelligence of artificial systems were to surpass that of humans significantly, this would constitute a significant risk for humanity. Time has come to consider these issues, and this consideration must include progress in AI as much as insights from the theory of AI. The papers in this volume try to make cautious headway in setting the problem, evaluating predictions on the future of AI, proposing ways to ensure that AI systems will be beneficial to humans – and critically (...) evaluating such proposals. (shrink)

There is, in some quarters, concern about high–level machine intelligence and superintelligent AI coming up in a few decades, bringing with it significant risks for humanity. In other quarters, these issues are ignored or considered science fiction. We wanted to clarify what the distribution of opinions actually is, what probability the best experts currently assign to high–level machine intelligence coming up within a particular time–frame, which risks they see with that development, and how fast they see these developing. We thus (...) designed a brief questionnaire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high-level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity. (shrink)

This is the editorial for a special volume of JETAI, featuring papers by Omohundro, Armstrong/Sotala/O’Heigeartaigh, T Goertzel, Brundage, Yampolskiy, B. Goertzel, Potapov/Rodinov, Kornai and Sandberg. - If the general intelligence of artificial systems were to surpass that of humans significantly, this would constitute a significant risk for humanity – so even if we estimate the probability of this event to be fairly low, it is necessary to think about it now. We need to estimate what progress we can expect, what (...) the impact of superintelligent machines might be, how we might design safe and controllable systems, and whether there are directions of research that should best be avoided or strengthened. (shrink)

Special Issue “Risks of artificial general intelligence”, Journal of Experimental and Theoretical Artificial Intelligence, 26/3 (2014), ed. Vincent C. Müller. http://www.tandfonline.com/toc/teta20/26/3# - Risks of general artificial intelligence, Vincent C. Müller, pages 297-301 - Autonomous technology and the greater human good - Steve Omohundro - pages 303-315 - - - The errors, insights and lessons of famous AI predictions – and what they mean for the future - Stuart Armstrong, Kaj Sotala & Seán S. Ó hÉigeartaigh - pages 317-342 - - (...) - The path to more general artificial intelligence - Ted Goertzel - pages 343-354 - - - Limitations and risks of machine ethics - Miles Brundage - pages 355-372 - - - Utility function security in artificially intelligent agents - Roman V. Yampolskiy - pages 373-389 - - - GOLEM: towards an AGI meta-architecture enabling both goal preservation and radical self-improvement - Ben Goertzel - pages 391-403 - - - Universal empathy and ethical bias for artificial general intelligence - Alexey Potapov & Sergey Rodionov - pages 405-416 - - - Bounding the impact of AGI - András Kornai - pages 417-438 - - - Ethics of brain emulations - Anders Sandberg - pages 439-457. (shrink)

[Müller, Vincent C. (ed.), (2013), Philosophy and theory of artificial intelligence (SAPERE, 5; Berlin: Springer). 429 pp. ] --- Can we make machines that think and act like humans or other natural intelligent agents? The answer to this question depends on how we see ourselves and how we see the machines in question. Classical AI and cognitive science had claimed that cognition is computation, and can thus be reproduced on other computing machines, possibly surpassing the abilities of human intelligence. This (...) consensus has now come under threat and the agenda for the philosophy and theory of AI must be set anew, re-defining the relation between AI and Cognitive Science. We can re-claim the original vision of general AI from the technical AI disciplines; we can reject classical cognitive science and replace it with a new theory (e.g. embodied); or we can try to find new ways to approach AI, for example from neuroscience or from systems theory. To do this, we must go back to the basic questions on computing, cognition and ethics for AI. The 30 papers in this volume provide cutting-edge work from leading researchers that define where we stand and where we should go from here. (shrink)

Invited papers from PT-AI 2011. - Vincent C. Müller: Introduction: Theory and Philosophy of Artificial Intelligence - Nick Bostrom: The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents - Hubert L. Dreyfus: A History of First Step Fallacies - Antoni Gomila, David Travieso and Lorena Lobo: Wherein is Human Cognition Systematic - J. Kevin O'Regan: How to Build a Robot that Is Conscious and Feels - Oron Shagrir: Computation, Implementation, Cognition.

The theory and philosophy of artificial intelligence has come to a crucial point where the agenda for the forthcoming years is in the air. This special volume of Minds and Machines presents leading invited papers from a conference on the “Philosophy and Theory of Artificial Intelligence” that was held in October 2011 in Thessaloniki. Artificial Intelligence is perhaps unique among engineering subjects in that it has raised very basic questions about the nature of computing, perception, reasoning, learning, language, action, interaction, (...) consciousness, humankind, life etc. etc. – and at the same time it has contributed substantially to answering these questions. There is thus a substantial tradition of work, both on AI by philosophers and of theory within AI itself. - The volume contains papers by Bostrom, Dreyfus, Gomila, O'Regan and Shagrir. (shrink)

Chalmers has articulated a compellingly simple argument for inevitability of the singularity—an explosion of increasingly intelligent machines, eventuating in super forms of intelligence. Chalmers then goes on to explore the implications of this outcome, and suggests ways in which we might prepare for the eventuality. I think Chalmers' argument proves both too much and too little. If the reasoning were right, it would follow inductively that the singularity already exists, in which case Chalmers would have proven more than he set (...) out to. Moreover, I will suggest that, if the singularity already exists, we are doomed. Fortunately, Chalmers' reasoning is problematic. I will consider several objections. Unfortunately, the most serious problem is that human life may end long before the singularity is created. In that case, we are doomed either way. Should we care? Perhaps not. (shrink)

The radical evolutionary step that divides human decision-making from that of nonhumans is the ability to excite the reward process for its own sake, in imagination. Combined with hyperbolic over-valuation of the present, this ability is a potential threat to both the individual's long term survival and the natural selection of high intelligence. Human belief is intrinsically or under-founded, which may or may not be adaptive.

Self-improvement was one of the aspects of AI proposed for study in the 1956 Dartmouth conference. Turing proposed a “child machine” which could be taught in the human manner to attain adult human-level intelligence. In latter days, the contention that an AI system could be built to learn and improve itself indefinitely has acquired the label of the bootstrap fallacy. Attempts in AI to implement such a system have met with consistent failure for half a century. Technological optimists, however, have (...) maintained that a such system is possible, producing, if implemented, a feedback loop that would lead to a rapid exponential increase in intelligence. We examine the arguments for both positions and draw some conclusions. (shrink)

The belief that computers will soon become transcendently intelligent and that human beings will “upload” their minds into machines has become ubiquitous in public discussions of robotics and artificial intelligence in Western cultures. Such beliefs are the result of pervasive Judaeo-Christian apocalyptic beliefs, and they have rapidly spread through modern pop and technological culture, including such varied and influential sources as Rolling Stone, the IEEE Spectrum, and official United States government reports. They have gained sufficient credibility to enable the construction (...) of Singularity University in California. While different approaches are possible (and, indeed, are common in Japan and possibly elsewhere), this particular vision of artificial intelligence and robotics has gained ground in the West through the influence of figures such as Hans Moravec and Ray Kurzweil. Because pop-science books help frame public discussion of new sciences and technologies for individuals, corporations, and governments alike, the integration of religious and technoscientific claims made by their authors should be clear and open for public and scientific debate. As we move forward into an increasingly robotic future, we should do so aware of the ways in which a group's religious environment can help set the tone for public acceptance and use of robotic technologies. (shrink)

Machine ethics and robot rights are quickly becoming hot topics in artificial intelligence and robotics communities. We will argue that attempts to attribute moral agency and assign rights to all intelligent machines are misguided, whether applied to infrahuman or superhuman AIs, as are proposals to limit the negative effects of AIs by constraining their behavior. As an alternative, we propose a new science of safety engineering for intelligent artificial agents based on maximizing for what humans value. In particular, we challenge (...) the scientific community to develop intelligent systems that have human-friendly values that they provably retain, even under recursive self-improvement. (shrink)

Are the robots coming? Is the singularity near? Will we be dominated by technology? The usual response to ethical issues raised by pervasive and ubiquitous technologies assumes a philosophical anthropology centered on existential autonomy and agency, a dualistic ontology separating humans from technology and the natural from the artificial, and a post-monotheistic dualist and creational spirituality. This paper explores an alternative, less modern vision of the “technological” future based on different assumptions: a “deep relational” view of human being and self, (...) an ecological view of human–technology relations, and “ubiquitous” spirituality. Moving beyond an ethics of fear and control, it is argued that technology is part of a lived and active whole that is at the same time human, technological, social, and spiritual. Influenced by ecological and Eastern thinking, it is concluded that an ethics of technology understood as a relational ethics of life asks us to adapt and grow within this multi-faced ecology, which is currently—but not necessarily—pervaded by hyper-individualist modernity and its ego-boosting technologies of the self. This growth is only possible by relating to, and learning from, other cultures and from their specific way of pervading and being pervaded. (shrink)

I show that the known fundamental laws of physics--quantum mechanics, general relativity, and the particle physics Standard Model -- imply that the Singularity will inevitably come to pass. Further, I show that there is an ethical system built into science and rationality itself -- thus the value-fact distinction is nonsense -- and this will preclude the AI's from destroying humanity even if they wished to do so. Finally, I show that the coming Singularity is good because only if it occurs (...) can life of any sort survive. Fortunately, the laws of physics, as I have said, require the Singularity to occur. If the laws of physics be for us, who can be against us? (shrink)

The present paper criticizes Chalmers's discussion of the Singularity, viewed as the emergence of a superhuman intelligence via the self-amplifying development of artificial intelligence. The situated and embodied view of cognition rejects the notion that intelligence could arise in a closed 'brain-in-a-vat' system, because intelligence is rooted in a high-bandwidth, sensory-motor interaction with the outside world. Instead, it is proposed that superhuman intelligence can emerge only in a distributed fashion, in the form of a self-organizing network of humans, computers, and (...) other technologies: the 'Global Brain'. (shrink)

In his 1993 article ‘The Coming Technological Singularity: How to survive in the posthuman era’ the computer scientist Virnor Vinge speculated that developments in artificial intelligence might reach a point where improvements in machine intelligence result in smart AI’s producing ever-smarter AI’s. According to Vinge the ‘singularity’, as he called this threshold of recursive self-improvement, would be a ‘transcendental event’ transforming life on Earth in ways that unaugmented humans are not equipped to envisage. In this paper I argue Vinge’s idea (...) of a technologically led intelligence explosion is philosophically important because it requires us to consider the prospect of a posthuman condition succeeding the human one. What is the ‘humanity’ to which the posthuman is ‘post’? Does the possibility of a posthumanity presuppose that there is a ‘human essence’, or is there some other way of conceiving the human-posthuman difference? I argue that the difference should be conceived as a historically emergent disconnection between individuals, not in terms of the presence or lack of essential properties. I also suggest that these individuals should not be conceived in narrow biological terms but in ‘wide’ terms permitting biological, cultural and technological relations of descent between human and posthuman. Finally, I consider the ethical implications of this metaphysics of the posthuman. If, as I claim, the posthuman difference is not one between kinds but between individuals, we cannot specify its nature a priori but only a posteriori. Thus the only way to evaluate the posthuman condition would be to witness the emergence of posthumans. The implications of this are somewhat paradoxical. We are not currently in a position to evaluate the posthuman condition. Since there are no posthumans, the condition of posthumanity is not defined. However, posthumans could result from some iteration of our current technical activity, so we have an interest in understanding what they might be like. It follows that we have an interest in making or becoming posthumans. (shrink)

Thought about the singularity intersects the philosophy of mind in deep and important ways. However, thought about the singularity also intersects many other areas of philosophy, including the history of philosophy, metaphysics, the philosophy of science, and the philosophy of religion. I point to some of those intersections. Singularitarian thought suggests that many of the objects and processes that once lay in the domain of revealed religion now lie in the domain of pure computer science.

The concept of a 'Singularity' is particularly intriguing as it is draws not just on philosophical but also neuroscientific issues. As a neuroscientist, perhaps my best contribution here therefore, would be to provide some reality checks against the elegant and challenging philosophical arguments set out by Chalmers. Aconvenient framework for addressing the points he raises will be to give my personal scientific take on the three basic questions summarised in the Conclusions section.

This paper attempts to formalize and to address the ‘leakproofing’ of the Singularity problem presented by David Chalmers. The paper begins with the definition of the Artificial Intelligence Confinement Problem. After analysis of existing solutions and their shortcomings, a protocol is proposed aimed at making a more secure confinement environment which might delay potential negative effect from the technological singularity while allowing humanity to benefit from the superintelligence.

Chalmers suggests that, if a Singularity fails to occur in the next few centuries, the most likely reason will be 'motivational defeaters' i.e. at some point humanity or human-level AI may abandon the effort to create dramatically superhuman artificial general intelligence. Here I explore one plausible way in which that might happen: the deliberate human creation of an 'AI Nanny' with mildly superhuman intelligence and surveillance powers, designed either to forestall Singularity eternally, or to delay the Singularity until humanity more (...) fully understands how to execute a Singularity in a positive way. It is suggested that as technology progresses, humanity may find the creation of an AI Nanny desirable as a means of protecting against the destructive potential of various advanced technologies such as AI, nanotechnology and synthetic biology. (shrink)