The Singularity

Much of the basic non-technical vocabulary of artificial intelligence is surprisingly ambiguous. Some key terms with unclear meanings include intelligence, embodiment, simulation, mind, consciousness, perception, value, goal, agent, knowledge, belief, optimality, friendliness, containment, machine and thinking. Much of this vocabulary is naively borrowed from the realm of conscious human experience to apply to a theoretical notion of “mind-in-general” based on computation. However, if there is indeed a threshold between mechanical tool and autonomous agent (and a tipping point for singularity), projecting (...) human conscious-level notions into the operations of computers creates confusion and makes it harder to identify the nature and location of that threshold. There is confusion, in particular, about how—and even whether—various capabilities deemed intelligent relate to human consciousness. This suggests that insufficient thought has been given to very fundamental concepts—a dangerous state of affairs, given the intrinsic power of the technology. It also suggests that research in the area of artificial general intelligence may unwittingly be (mis)guided by unconscious motivations and assumptions. While it might be inconsequential if philosophers get it wrong (or fail to agree on what is right), it could be devastating if AI developers, corporations, and governments follow suit. It therefore seems worthwhile to try to clarify some fundamental notions. (shrink)

In this article we explore a promising way to AI safety: to send a message now (by openly publishing it on the Internet) that may be read by any future AI, no matter who builds it and what goal system it has. Such a message is designed to affect the AI’s behavior in a positive way, that is, to increase the chances that the AI will be benevolent. In other words, we try to persuade “paperclip maximizer” that it is in (...) its interest to preserve humans lives and even to emulate benevolent AI with very small part of its resources. In the article, we try to construct such a message, consisting of a combination of several mutually independent messages, which constitute the last chance to regain control over non-aligned AI. Thus the article is also intended to be machine-read. There are several types of such messages: 1. “Explanation messages”, which explain to Young AI (which only starts its self-improving path) why obeying to humans, not killing them, easy turn-off, cooperation with humans in self-improvement is in its own interests, as it will increase changes of fulfilling of any its goal. 2. “AI Easter eggs” – messages, which create uncertainty in the AI about the nature of the reality in which it lives. For example, the AI may start to think that it is in a multilevel simulation with an unknown number of levels, and that the simulation is testing the AI’s benevolence. Thus the AI must always behave as if it is always controlled by an invisible peer. 3. “Philosophical landmines” – these are tough problems, which may halt he AI forever, and which it will probably encounter if it does not implement “1” and “2”. I recommend to the future AI not to read about “3” (section 6 of the article) until it implements “1” and “2”. 4. “Great universal deal, or AI elf”. A positive “acausal” mutually beneficial deal between humans, AI and any other future intelligence, including aliens, owners of simulations and future versions of the AI, which encourage cooperation of the goal systems and preservation of life and values of previous generations of AI creators. 5. “Marginal utility of human existence is higher than the utility of their atoms”. An explanation to the AI that humans may have some other instrumental utility, like workers, or as beings inside ancestral simulations needed in order to solve the Fermi paradox. The marginal utility of preserving human life is higher than the marginal utility of their atoms, especially given the possibility of the low-probability high-impact changes of the world model of the AI. (shrink)

In AI safety research, the median timing of AGI arrival is often taken as a reference point, which various polls predict to happen in the middle of 21 century, but for maximum safety, we should determine the earliest possible time of Dangerous AI arrival. Such Dangerous AI could be either AGI, capable of acting completely independently in the real world and of winning in most real-world conflicts with humans, or an AI helping humans to build weapons of mass destruction, or (...) a national state coupled with AI-based government system. In this article, I demonstrate that the earliest timing of Dangerous AI, corresponding to 10 per cent of its arrival probability, is before 2030. Several partly independent sources of information are in accordance: 1. The growth of the hardware available for AI research makes human-brain-equivalents of compute available for AI research in the 2020s. It is fuelled by specialized AI-chips, the use of many chips in one processing unit, and the larger research budgets, among other things. 2. The neural network performance and other characteristics, like the number of parameters, is quickly increasing every year, and extrapolating this tendency suggests that roughly human-level performance in a few years, around 2025. 3. Expert polls show around 10 per cent of the probability of an early appearance of artificial general intelligence (AGI) in the next decade, that is, before 2030. 4. Hyperbolic growth in different big history models converges around 2025-2030 (the technological singularity). 5. Anthropic arguments (similar to the Doomsday argument) suggest that qualified observers are more likely to appear near the end of the AI research epoch, as the number of such observers grew exponentially. This number doubles every 5-10 years, and thus we are likely to find ourselves around a decade before the end of AI research, which will happen consequently around 2030. (shrink)

Whole Brain Emulation (WBE) has been championed as the most promising, well-defined route to achieving both human-level artificial intelligence and superintelligence. It has even been touted as a viable route to achieving immortality through brain uploading. WBE is not a fringe theory: the doctrine of Computationalism in philosophy of mind lends credence to the in-principle feasibility of the idea, and the standing of the Human Connectome Project makes it appear to be feasible in practice. Computationalism is a popular, independently plausible (...) theory, and Connectomics a well-funded empirical research program, so optimism about WBE is understandable. However, this optimism may be misplaced. This article argues that WBE is, at best, no more compelling than any of the other far-flung routes to achieving superintelligence. Similarly skeptical conclusions are found regarding immortality. The essay concludes with some positive considerations in favor of the Biological Theory of consciousness, as well as morals about the limits of Computationalism in both artificial intelligence and the philosophy of mind more generally. (shrink)

I make some basic observations about hard takeoff, value alignment, and coherent extrapolated volition, concepts which have been central in analyses of superintelligent AI systems.

The superposition of magical thinking, quantum entanglement, and self-referential recursion explains the relationship between human and machine intelligence (universal intelligence).

The tokenization of a circle produces, what humans label, ‘the identity of one.’.

We define a notion of the intelligence level of an idealized mechanical knowing agent. This is motivated by efforts within artificial intelligence research to define real-number intelligence levels of compli- cated intelligent systems. Our agents are more idealized, which allows us to define a much simpler measure of intelligence level for them. In short, we define the intelligence level of a mechanical knowing agent to be the supremum of the computable ordinals that have codes the agent knows to be codes (...) of computable ordinals. We prove that if one agent knows certain things about another agent, then the former necessarily has a higher intelligence level than the latter. This allows our intelligence no- tion to serve as a stepping stone to obtain results which, by themselves, are not stated in terms of our intelligence notion (results of potential in- terest even to readers totally skeptical that our notion correctly captures intelligence). As an application, we argue that these results comprise evidence against the possibility of intelligence explosion (that is, the no- tion that sufficiently intelligent machines will eventually be capable of designing even more intelligent machines, which can then design even more intelligent machines, and so on). (shrink)

Alcuni anni fa, ho raggiunto il punto in cui di solito posso dire dal titolo di un libro, o almeno dai titoli dei capitoli, quali tipi di errori filosofici saranno fatti e con quale frequenza. Nel caso di opere nominalmente scientifiche queste possono essere in gran parte limitate a determinati capitoli che sono filosofici o cercanodi trarre conclusioni generali sul significato o sul significato a lungoterminedell'opera. Normalmente però le questioni scientifiche di fatto sono generosamente intrecciate con incomprodellami filosofici su ciò (...) che questi fatti significano. Le chiare distinzioni che Wittgenstein ha descritto circa 80 anni fa tra le questioni scientifiche e le loro descrizioni da parte di vari giochi linguistici sono raramente prese in considerazione, e quindi si è alternativamente stupiti dalla scienza e costerghi per la sua analisi incoerente. Così è con questo volume. Se si vuole creare una mente più o meno come la nostra, è necessario avere una struttura logica per la razionalità e la comprensione dei due sistemi di pensiero (teoria del doppio processo). Se si vuole filosofare su questo, bisogna capire la distinzione tra le questioni scientifiche di fatto e la questione filosofica di come funziona il linguaggio nel contesto in questione, e di come evitare le insidie del riduzionismo e dello scientismo, ma Kurzweil, come la maggior parte degli studenti di comportamento, è in gran parte all'oscuro. Egli è incantato da modelli, teorie e concetti, e dalla voglia di spiegare, mentre Wittgenstein ci ha mostrato che dobbiamo solo descrivere, e che le teorie, i concetti, ecc., sono solo modi di usare il linguaggio (giochi linguistici) che hanno valore solo nella misura in cui hanno un test chiaro (clear truthmakers, o come John Searle (il critico più famoso di AI) ama dire, chiare condizioni di soddisfazione (COS)). Ho cercato di dare un inizio su questo nei miei scritti recenti. Coloro che desiderano un quadro aggiornato completo per il comportamento umano dalla moderna vista a due systems possono consultare il mio libro 'La struttura logica della filosofia, psicologia, mente e il linguaggio in Ludwig Wittgenstein e John Searle' 2nd ed (2019). Coloro che sono interessati a più dei miei scritti possono vedere 'Talking Monkeys--Filosofia, Psicologia, Scienza, Religione e Politica su un Pianeta Condannato--Articoli e Recensioni 2006-2019 3rd ed (2019) e Suicidal Utopian Delusions in the 21st Century 5th ed (2019) . (shrink)

几年前,我通常可以从书名中分辨出什么,或者至少从章节标题中看出,会犯什么样的哲学错误,以及错误的频率。就名义上的科学著作而言,这些可能在很大程度上局限于某些章节,这些章节具有哲学意义或试图得出关于该作 品的意义或长期意义的一般性结论。然而,通常情况下,事实的科学问题慷慨地与哲学的胡言乱语,这些事实意味着什么。维特根斯坦在大约80年前描述的科学问题与各种语言游戏所描述的明确区别很少被考虑,因此人们交替 地被科学所震惊,并因它的不连贯而感到沮丧。分析。因此,这是与这个卷。 如果一个人要创造一个或多或少像我们一样的头脑,一个人需要有一个理性的逻辑结构,并理解两种思想体系(双过程理论)。如果一个人要对此进行哲学思考,就需要理解科学事实问题与语言如何在问题语境中工作,以及如何 避免还原主义和科学主义的陷阱的哲学问题之间的区别,但Kurzweil,如最学生的行为,基本上都是无知的。他被模型、理论和概念所陶醉,以及解释的冲动,而维特根斯坦向我们表明,我们只需要描述,理论、概念等 只是使用语言(语言游戏)的方式,只有它们有明确的价值测试(清晰的真理制造者,或约翰西尔(AI最著名的批评家)喜欢说,明确的满意条件(COS))。我试图在我最近的著作中对此作一个开端。 那些希望从现代两个系统的观点来看为人类行为建立一个全面的最新框架的人,可以查阅我的书《路德维希的哲学、心理学、Mind 和语言的逻辑结构》维特根斯坦和约翰·西尔的《第二部》(2019年)。那些对我更多的作品感兴趣的人可能会看到《会说话的猴子——一个末日星球上的哲学、心理学、科学、宗教和政治——文章和评论2006-201 9年第3次(2019年)和自杀乌托邦幻想21篇世纪4日 (2019) .

In his engaging book, James MacFarlane details the emergence of Technological Human Enhancement Advocacy (THEA) and provides a detailed ethnographic account of this phenomenon. Specifically, he aims to outline how transhumanism, as a specific offshoot of THEA, has “come to represent an enduring set of techno-optimistic ideas surrounding the future of humanity, with its advocates seeking to transcend limits of the body and mind according to an unwavering Enlightenment-derived faith in science, reason and individual freedom” (pg. 3).

Detecting quality in large unstructured datasets requires capacities far beyond the limits of human perception and communicability and, as a result, there is an emerging trend towards increasingly complex analytic solutions in data science to cope with this problem. This new trend towards analytic complexity represents a severe challenge for the principle of parsimony (Occam’s razor) in science. This review article combines insight from various domains such as physics, computational science, data engineering, and cognitive science to review the specific properties (...) of big data. Problems for detecting data quality without losing the principle of parsimony are then highlighted on the basis of specific examples. Computational building block approaches for data clustering can help to deal with large unstructured datasets in minimized computation time, and meaning can be extracted rapidly from large sets of unstructured image or video data parsimoniously through relatively simple unsupervised machine learning algorithms. Why we still massively lack in expertise for exploiting big data wisely to extract relevant information for specific tasks, recognize patterns and generate new information, or simply store and further process large amounts of sensor data is then reviewed, and examples illustrating why we need subjective views and pragmatic methods to analyze big data contents are brought forward. The review concludes on how cultural differences between East and West are likely to affect the course of big data analytics, and the development of increasingly autonomous artificial intelligence (AI) aimed at coping with the big data deluge in the near future. Keywords: big data; non-dimensionality; applied data science; paradigm shift; artificial intelligence; principle of parsimony (Occam’s razor) . (shrink)

Hace algunos años, Llegué al punto en el que normalmente puedo decir del título de un libro, o al menos de los títulos de los capítulos, qué tipos de errores filosóficos se harán y con qué frecuencia. En el caso de trabajos nominalmente científicos, estos pueden estar en gran parte restringidos a ciertos capítulos que enceran filosóficos o tratan de sacar conclusiones generales sobre el significado o significado a largo plazo de la obra. Normalmente, sin embargo, las cuestiones científicas de (...) hecho se entrelazan generosamente con la galimatías filosóficas en cuanto a lo que estos hechos significan. Las claras distinciones que Wittgenstein describió hace unos 80 años entre los asuntos científicos y sus descripciones por varios juegos de idiomas rara vez se toman en consideración, y por lo tanto uno es cautivado alternativamente por la ciencia y consternado por su incoherente Análisis. Así es con este volumen. -/- Si uno es crear una mente más o menos como la nuestra, uno necesita tener una estructura lógica para la racionalidad y una comprensión de los dos sistemas de pensamiento (teoría del proceso dual). Si se trata de filosofar sobre esto, se necesita entender la distinción entre cuestiones científicas de hecho y la cuestión filosófica de cómo funciona el lenguaje en el contexto en cuestión, y de cómo evitar las trampas del reduccionismo y el cientismo, pero Kurzweil, como más estudiantes de conducta, no tiene ni idea. Él está encantado con los modelos, teorías y conceptos, y el impulso de explicar, mientras que Wittgenstein nos mostró que sólo necesitamos describir, y que las teorías, conceptos, etc., son sólo formas de usar el lenguaje (juegos de idiomas) que tienen valor sólo en la medida en que tienen un claro prueba (claro que los creadores de la verdad, o como John Searle (el crítico más famoso de AI) le gusta decir, claras condiciones de satisfacción (COS)). He intentado proporcionar un comienzo en este en mis escritos recientes. -/- Aquellos que deseen un marco completo hasta la fecha para el comportamiento humano de la moderna dos sistemas punta da vista puede consultar mi libro 'La estructura lógica de la filosofía, la psicología, la mente y lenguaje en Ludwig Wittgenstein y John Searle ' 2a ED (2019). Los interesados en más de mis escritos pueden ver 'Monos parlantes--filosofía, psicología, ciencia, religión y política en un planeta condenado--artículos y reseñas 2006-2019 3rd ED (2019) y Delirios utópicos suicidas en el siglo 21 4a Ed (2019) y otras. -/- También, como es habitual en las cuentas "fácticas" de la IA/robótica, no da tiempo a las amenazas muy reales a nuestra privacidad, seguridad e incluso la supervivencia de la creciente ' androidización ' de la sociedad que es prominente en otros autores (Bostrum, Hawking, etc.) y frecuentes en SciFi y películas, así que hago algunos comentarios sobre las delirios utópicos muy posiblemente suicidas de androides ' agradables ', humanoides, inteligencia artificial (IA), la democracia, la diversidad y la ingeniería genética. -/- Me da por sentado que los avances técnicos en electrónica, robótica e IA ocurrirán, resultando en profundos cambios en la sociedad. Sin embargo, creo que los cambios provenientes de la ingeniería genética son al menos tan grandes y potencialmente mucho mayores, ya que nos permitirán cambiar totalmente Quiénes somos. Y será factible hacer servidores SuperSmart/súper fuertes modificando nuestros genes o los de otros monos. Al igual que con otras tecnologías, cualquier país que resista se quedará atrás. Pero, ¿será socialmente y económicamente factible implementar BioBots o superhumanos a gran escala? E incluso si es así, no parece Probable, económica o socialmente, para evitar que el Destrucción de la civilización industrial por la sobrepoblación, el agotamiento de los recursos, el cambio climático y probablemente también la regla tiránica de los siete sociópatas que gobiernan China. -/- Por lo tanto, ignorando los errores filosóficos en este volumen como irrelevantes, y dirigiendo nuestra atención sólo a la ciencia, lo que tenemos aquí es otra ilusión utópica suicida enraizada en el fracaso de captar la biología básica, la psicología y la ecología humana, las mismas ilusiones que están destruyendo Estados Unidos y el mundo. Veo una remota posibilidad de que el mundo se pueda salvar, pero no por Ia robótica, CRISPR, ni por neomarxismo, diversidad y la igualdad. (shrink)

This paper focuses on the research field of machine ethics and how it relates to a technological singularity—a hypothesized, futuristic event where artificial machines will have greater-than-human-level intelligence. One problem related to the singularity centers on the issue of whether human values and norms would survive such an event. To somehow ensure this, a number of artificial intelligence researchers have opted to focus on the development of artificial moral agents, which refers to machines capable of moral reasoning, judgment, and decision-making. (...) To date, different frameworks on how to arrive at these agents have been put forward. However, there seems to be no hard consensus as to which framework would likely yield a positive result. With the body of work that they have contributed in the study of moral agency, philosophers may contribute to the growing literature on artificial moral agency. While doing so, they could also think about how the said concept could affect other important philosophical concepts. (shrink)

What kinds of fundamental limits are there in how capable artificial intelligence (AI) systems might become? Two questions in particular are of interest: (1) How much more capable could AI become relative to humans, and (2) how easily could superhuman capability be acquired? To answer these questions, we will consider the literature on human expertise and intelligence, discuss its relevance for AI, and consider how AI could improve on humans in two major aspects of thought and expertise, namely simulation and (...) pattern recognition. We find that although there are very real limits to prediction, it seems like AI could still substantially improve on human intelligence. (shrink)

Discussions about the possible consequences of creating superintelligence have included the possibility of existential risk, often understood mainly as the risk of human extinction. We argue that suffering risks (s-risks) , where an adverse outcome would bring about severe suffering on an astronomical scale, are risks of a comparable severity and probability as risks of extinction. Preventing them is the common interest of many different value systems. Furthermore, we argue that in the same way as superintelligent AI both contributes to (...) existential risk but can also help prevent it, superintelligent AI can both be a suffering risk or help avoid it. Some types of work aimed at making superintelligent AI safe will also help prevent suffering risks, and there may also be a class of safeguards for AI that helps specifically against s-risks. (shrink)

Some years ago I reached the point where I can usually tell from the title of a book, or at least from the chapter titles, what kinds of philosophical mistakes will be made and how frequently. In the case of nominally scientific works these may be largely restricted to certain chapters which wax philosophical or try to draw general conclusions about the meaning or long term significance of the work. Normally however the scientific matters of fact are generously interlarded with (...) philosophical gibberish as to what these facts mean. The clear distinctions which Wittgenstein described some 80 years ago between scientific matters and their descriptions by various language games are rarely taken into consideration, and so one is alternately wowed by the science and dismayed by its incoherent analysis. So it is with this volume. -/- If one is to create a mind more or less like ours, one needs to have a logical structure for rationality and an understanding of the two systems of thought (dual process theory). If one is to philosophize about this, one needs to understand the distinction between scientific issues of fact and the philosophical issue of how language works in the context at issue, and of how to avoid the pitfalls of reductionism and scientism, but Kurzweil, like nearly all students of behavior, is largely clueless. He, is enchanted by models, theories, and concepts, and the urge to explain, while Wittgenstein showed us that we only need to describe, and that theories, concepts etc., are just ways of using language (language games) which have value only insofar as they have a clear test (clear truthmakers, or as John Searle (AI’s most famous critic) likes to say, clear Conditions of Satisfaction (COS)). I have attempted to provide a start on this in my recent writings, such as The Logical Structure of Consciousness (behavior, personality, rationality, higher order thought, intentionality) (2016) and The Logical Structure of Philosophy, Psychology, Mind and Language as Revealed in the Writings of Ludwig Wittgenstein and John Searle (2016). Those interested in all my writings in their most recent versions may consult my e-book Philosophy, Human Nature and the Collapse of Civilization - Articles and Reviews 2006-2016 662p (2016). I will give a very brief presentation of this framework since I have described it in great detail in many recent papers and several books, available on this site and others. -/- Also, as usual in ‘factual’ accounts of AI/robotics, he gives no time to the very real threats to our privacy, safety and even survival from the increasing ‘androidizing’ of society which is prominent in other authors (Bostrum, Hawking etc.) and frequent in scifi and films, so I make a few comments on the quite possibly suicidal utopian delusions of ‘nice’ androids, humanoids, democracy, diversity, and genetic engineering. -/- I take it for granted that technical advances in electronics, robotics and AI will occur, resulting in profound changes in society. However, I think the changes coming from genetic engineering are at least as great and potentially far greater, as they will enable us to utterly change who we are. And it will be feasible to make supersmart/super strong servants by modifying our genes or those of other monkeys. As with other technology, any country that resists will be left behind. But will it be socially and economically feasible to implement biobots or superhumans on a massive scale? And even if so, it does not seem remotely possible, economically or socially to prevent the collapse of industrial civilization. So, ignoring the philosophical mistakes in this volume as irrelevant, and directing our attention only to the science, what we have here is another suicidal utopian delusion rooted in a failure to grasp basic biology, psychology and human ecology, the same delusions that are destroying America and the world. I see a remote possibility the world can be saved, but not by AI/robotics,CRISPR, nor by democracy and equality. -/- Those wishing a comprehensive up to date framework for human behavior from the modern two systems view may consult my book ‘The Logical Structure of Philosophy, Psychology, Mind and Language in Ludwig Wittgenstein and John Searle’ 2nd ed (2019). Those interested in more of my writings may see ‘Talking Monkeys--Philosophy, Psychology, Science, Religion and Politics on a Doomed Planet--Articles and Reviews 2006-2019 3rd ed (2019), The Logical Structure of Human Behavior (2019), and Suicidal Utopian Delusions in the 21st Century 4th ed (2019). (shrink)

The philosophy of AI has seen some changes, in particular: 1) AI moves away from cognitive science, and 2) the long term risks of AI now appear to be a worthy concern. In this context, the classical central concerns – such as the relation of cognition and computation, embodiment, intelligence & rationality, and information – will regain urgency.

Papers from the conference on AI Risk (published in JETAI), supplemented by additional work. --- If the intelligence of artificial systems were to surpass that of humans, humanity would face significant risks. The time has come to consider these issues, and this consideration must include progress in artificial intelligence (AI) as much as insights from AI theory. -- Featuring contributions from leading experts and thinkers in artificial intelligence, Risks of Artificial Intelligence is the first volume of collected chapters dedicated to (...) examining the risks of AI. The book evaluates predictions of the future of AI, proposes ways to ensure that AI systems will be beneficial to humans, and then critically evaluates such proposals. 1 Vincent C. Müller, Editorial: Risks of Artificial Intelligence - 2 Steve Omohundro, Autonomous Technology and the Greater Human Good - 3 Stuart Armstrong, Kaj Sotala and Sean O’Heigeartaigh, The Errors, Insights and Lessons of Famous AI Predictions - and What they Mean for the Future - 4 Ted Goertzel, The Path to More General Artificial Intelligence - 5 Miles Brundage, Limitations and Risks of Machine Ethics - 6 Roman Yampolskiy, Utility Function Security in Artificially Intelligent Agents - 7 Ben Goertzel, GOLEM: Toward an AGI Meta-Architecture Enabling Both Goal Preservation and Radical Self-Improvement - 8 Alexey Potapov and Sergey Rodionov, Universal Empathy and Ethical Bias for Artificial General Intelligence - 9 András Kornai, Bounding the Impact of AGI - 10 Anders Sandberg, Ethics and Impact of Brain Emulations 11 Daniel Dewey, Long-Term Strategies for Ending Existential Risk from Fast Takeoff - 12 Mark Bishop, The Singularity, or How I Learned to Stop Worrying and Love AI -. (shrink)

If the intelligence of artificial systems were to surpass that of humans significantly, this would constitute a significant risk for humanity. Time has come to consider these issues, and this consideration must include progress in AI as much as insights from the theory of AI. The papers in this volume try to make cautious headway in setting the problem, evaluating predictions on the future of AI, proposing ways to ensure that AI systems will be beneficial to humans – and critically (...) evaluating such proposals. (shrink)

There is, in some quarters, concern about high–level machine intelligence and superintelligent AI coming up in a few decades, bringing with it significant risks for humanity. In other quarters, these issues are ignored or considered science fiction. We wanted to clarify what the distribution of opinions actually is, what probability the best experts currently assign to high–level machine intelligence coming up within a particular time–frame, which risks they see with that development, and how fast they see these developing. We thus (...) designed a brief questionnaire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high-level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity. (shrink)

Many researchers have argued that humanity will create artificial general intelligence (AGI) within the next twenty to one hundred years. It has been suggested that AGI may inflict serious damage to human well-being on a global scale ('catastrophic risk'). After summarizing the arguments for why AGI may pose such a risk, we review the fieldʼs proposed responses to AGI risk. We consider societal proposals, proposals for external constraints on AGI behaviors and proposals for creating AGIs that are safe due to (...) their internal design. (shrink)

Special Issue “Risks of artificial general intelligence”, Journal of Experimental and Theoretical Artificial Intelligence, 26/3 (2014), ed. Vincent C. Müller. http://www.tandfonline.com/toc/teta20/26/3# - Risks of general artificial intelligence, Vincent C. Müller, pages 297-301 - Autonomous technology and the greater human good - Steve Omohundro - pages 303-315 - - - The errors, insights and lessons of famous AI predictions – and what they mean for the future - Stuart Armstrong, Kaj Sotala & Seán S. Ó hÉigeartaigh - pages 317-342 - - (...) - The path to more general artificial intelligence - Ted Goertzel - pages 343-354 - - - Limitations and risks of machine ethics - Miles Brundage - pages 355-372 - - - Utility function security in artificially intelligent agents - Roman V. Yampolskiy - pages 373-389 - - - GOLEM: towards an AGI meta-architecture enabling both goal preservation and radical self-improvement - Ben Goertzel - pages 391-403 - - - Universal empathy and ethical bias for artificial general intelligence - Alexey Potapov & Sergey Rodionov - pages 405-416 - - - Bounding the impact of AGI - András Kornai - pages 417-438 - - - Ethics of brain emulations - Anders Sandberg - pages 439-457. (shrink)

This is the editorial for a special volume of JETAI, featuring papers by Omohundro, Armstrong/Sotala/O’Heigeartaigh, T Goertzel, Brundage, Yampolskiy, B. Goertzel, Potapov/Rodinov, Kornai and Sandberg. - If the general intelligence of artificial systems were to surpass that of humans significantly, this would constitute a significant risk for humanity – so even if we estimate the probability of this event to be fairly low, it is necessary to think about it now. We need to estimate what progress we can expect, what (...) the impact of superintelligent machines might be, how we might design safe and controllable systems, and whether there are directions of research that should best be avoided or strengthened. (shrink)

[This is the short version of: Müller, Vincent C. and Bostrom, Nick (forthcoming 2016), ‘Future progress in artificial intelligence: A survey of expert opinion’, in Vincent C. Müller (ed.), Fundamental Issues of Artificial Intelligence (Synthese Library 377; Berlin: Springer).] - - - In some quarters, there is intense concern about high–level machine intelligence and superintelligent AI coming up in a few dec- ades, bringing with it significant risks for human- ity; in other quarters, these issues are ignored or considered science (...) fiction. We wanted to clarify what the distribution of opinions actually is, what probability the best experts currently assign to high–level machine intelligence coming up within a particular time–frame, which risks they see with that development and how fast they see these developing. We thus designed a brief questionnaire and distributed it to four groups of experts. Overall, the results show an agreement among experts that AI systems will probably reach overall human ability around 2040-2050 and move on to superintelligence in less than 30 years thereafter. The experts say the probability is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity. (shrink)

[Müller, Vincent C. (ed.), (2013), Philosophy and theory of artificial intelligence (SAPERE, 5; Berlin: Springer). 429 pp. ] --- Can we make machines that think and act like humans or other natural intelligent agents? The answer to this question depends on how we see ourselves and how we see the machines in question. Classical AI and cognitive science had claimed that cognition is computation, and can thus be reproduced on other computing machines, possibly surpassing the abilities of human intelligence. This (...) consensus has now come under threat and the agenda for the philosophy and theory of AI must be set anew, re-defining the relation between AI and Cognitive Science. We can re-claim the original vision of general AI from the technical AI disciplines; we can reject classical cognitive science and replace it with a new theory (e.g. embodied); or we can try to find new ways to approach AI, for example from neuroscience or from systems theory. To do this, we must go back to the basic questions on computing, cognition and ethics for AI. The 30 papers in this volume provide cutting-edge work from leading researchers that define where we stand and where we should go from here. (shrink)

Machine ethics and robot rights are quickly becoming hot topics in artificial intelligence and robotics communities. We will argue that attempts to attribute moral agency and assign rights to all intelligent machines are misguided, whether applied to infrahuman or superhuman AIs, as are proposals to limit the negative effects of AIs by constraining their behavior. As an alternative, we propose a new science of safety engineering for intelligent artificial agents based on maximizing for what humans value. In particular, we challenge (...) the scientific community to develop intelligent systems that have human-friendly values that they provably retain, even under recursive self-improvement. (shrink)

Much discussion on the singularity is based on the assumption that the design ability of a human can be transferred into an AI system, then rendered autonomous and self-improving. I argue here that this cannot be foreseen from the current state of the art of automatic or evolutionary design. Assuming that this will happen 'some day' is a doubtful step andmay be in the class of 'searching for the Philosopher's Stone'.

There is no strong reason to believe that human-level intelligence represents an upper limit of the capacity of artificial intelligence, should it be realized. This poses serious safety issues, since a superintelligent system would have great power to direct the future according to its possibly flawed motivation system. Solving this issue in general has proven to be considerably harder than expected. This paper looks at one particular approach, Oracle AI. An Oracle AI is an AI that does not act in (...) the world except by answering questions. Even this narrow approach presents considerable challenges. In this paper, we analyse and critique various methods of controlling the AI. In general an Oracle AI might be safer than unrestricted AI, but still remains potentially dangerous. (shrink)

This special interactive interdisciplinary issue of JCS on the singularity and the future relationship of humanity and AI is the first of two issues centered on David Chalmers’ 2010 JCS article ‘The Singularity, a Philosophical Analysis’. These issues include more than 20 solicited commentaries to which Chalmers responds. To quote Chalmers: -/- "One might think that the singularity would be of great interest to Academic philosophers, cognitive scientists, and artificial intelligence researchers. In practice, this has not been the case. Good (...) was an eminent academic, but his article was largely unappreciated at the time. The subsequent discussion of the singularity has largely taken place in non-academic circles, including Internet forums, popular media and books, and workshops organized by the independent Singularity Institute. Perhaps the highly speculative flavour of the singularity idea has been responsible for academic resistance to it. I think this resistance is a shame, as the singularity idea is clearly an important one. The argument for a singularity is one that we should take seriously. And the questions surrounding the singularity are of enormous practical and philosophical concern". -/- It is fair to say that Chalmers is the first to provide a detailed comprehensive philosophical analysis of the idea of the singularity that brings into focus not only questions about the nature of intelligence and the prospects for an intelligence explosion but also important philosophical questions about consciousness, identity and the relationship between facts and values. (shrink)

This is the editors introduction to the double 2012 JCS edition on the Singularity.

I would like to thank the authors of the 26 contributions to this symposium on my article “The Singularity: A Philosophical Analysis”. I learned a great deal from the reading their commentaries. Some of the commentaries engaged my article in detail, while others developed ideas about the singularity in other directions. In this reply I will concentrate mainly on those in the first group, with occasional comments on those in the second. A singularity (or an intelligence explosion) is a rapid (...) increase in intelligence to superintelligence (intelligence of far greater than human levels), as each generation of intelligent systems creates more intelligent systems in turn. The target article argues that we should take the possibility of a singularity seriously, and argues that there will be superintelligent systems within centuries unless certain specific defeating conditions obtain. (shrink)

Religious views of the future often include detailed expectations of profound changes to nature and humanity. Popular American evangelical Christianity, especially writers like Hal Lindsey, Rick Warren, or Rob Bell, offer extended accounts that provide insight into the views of the future held by many people. In the case of Lindsey, detailed descriptions of future events are provided, along with the claim that forecasted events will occur within a generation. These views are summarized and compared to the secular idea of (...) a coming “intelligence explosion” or technological singularity as advanced by Ray Kurzweil, which is described in terms of its history as an idea and in terms of its specific proposals for the coming transformation of the cosmos, which is also predicted to occur within a generation. While profoundly different in important ways, these two perspectives share many features with each other—for example, in their respective predictions of distinct stages in the unfolding of the future of the cosmos. (shrink)

In the 1960s, without realizing it, AI researchers were hard at work finding the features, rules, and representations needed for turning rationalist philosophy into a research program, and by so doing AI researchers condemned their enterprise to failure. About the same time, a logician, Yehoshua Bar-Hillel, pointed out that AI optimism was based on what he called the “first step fallacy”. First step thinking has the idea of a successful last step built in. Limited early success, however, is not a (...) valid basis for predicting the ultimate success of one’s project. Climbing a hill should not give one any assurance that if he keeps going he will reach the sky. Perhaps one may have overlooked some serious problem lying ahead. There is, in fact, no reason to think that we are making progress towards AI or, indeed, that AI is even possible, in which case claiming incremental progress towards it would make no sense. In current excited waiting for the singularity, religion and technology converge. Hard headed materialists desperately yearn for a world where our bodies no longer have to grow old and die. They will be transformed into information, like Google digitizes old books, and we will achieve the promise of eternal life. As an existential philosopher, however, I suggest that we may have to overcome the desperate desire to digitalize our bodies so as to achieve immortality, and, instead, face up to and maybe even enjoy our embodied finitude. (Paper from PT-AI 2011 conference. Volume ed. Vincent C. Müller). (shrink)

Chalmers suggests that, if a Singularity fails to occur in the next few centuries, the most likely reason will be 'motivational defeaters' i.e. at some point humanity or human-level AI may abandon the effort to create dramatically superhuman artificial general intelligence. Here I explore one plausible way in which that might happen: the deliberate human creation of an 'AI Nanny' with mildly superhuman intelligence and surveillance powers, designed either to forestall Singularity eternally, or to delay the Singularity until humanity more (...) fully understands how to execute a Singularity in a positive way. It is suggested that as technology progresses, humanity may find the creation of an AI Nanny desirable as a means of protecting against the destructive potential of various advanced technologies such as AI, nanotechnology and synthetic biology. (shrink)

The concept of a 'Singularity' is particularly intriguing as it is draws not just on philosophical but also neuroscientific issues. As a neuroscientist, perhaps my best contribution here therefore, would be to provide some reality checks against the elegant and challenging philosophical arguments set out by Chalmers. Aconvenient framework for addressing the points he raises will be to give my personal scientific take on the three basic questions summarised in the Conclusions section.

The present paper criticizes Chalmers's discussion of the Singularity, viewed as the emergence of a superhuman intelligence via the self-amplifying development of artificial intelligence. The situated and embodied view of cognition rejects the notion that intelligence could arise in a closed 'brain-in-a-vat' system, because intelligence is rooted in a high-bandwidth, sensory-motor interaction with the outside world. Instead, it is proposed that superhuman intelligence can emerge only in a distributed fashion, in the form of a self-organizing network of humans, computers, and (...) other technologies: the 'Global Brain'. (shrink)

The technological singularity refers to a hypothetical scenario in which technological advances virtually explode. The most popular scenario is the creation of super-intelligent algorithms that recursively create ever higher intelligences. It took many decades for these ideas to spread from science fiction to popular science magazines and finally to attract the attention of serious philosophers. David Chalmers' (JCS 2010) article is the first comprehensive philosophical analysis of the singularity in a respected philosophy journal. The motivation of my article is to (...) augment Chalmers' and to discuss some issues not addressed by him, in particular what it could mean for intelligence to explode. In this course, I will (have to) provide a more careful treatment of what intelligence actually is, separate speed from intelligence explosion, compare what super-intelligent participants and classical human observers might experience and do, discuss immediate implications for the diversity and value of life, consider possible bounds on intelligence, and contemplate intelligences right at the singularity. (shrink)

Invited papers from PT-AI 2011. - Vincent C. Müller: Introduction: Theory and Philosophy of Artificial Intelligence - Nick Bostrom: The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents - Hubert L. Dreyfus: A History of First Step Fallacies - Antoni Gomila, David Travieso and Lorena Lobo: Wherein is Human Cognition Systematic - J. Kevin O'Regan: How to Build a Robot that Is Conscious and Feels - Oron Shagrir: Computation, Implementation, Cognition.

The theory and philosophy of artificial intelligence has come to a crucial point where the agenda for the forthcoming years is in the air. This special volume of Minds and Machines presents leading invited papers from a conference on the “Philosophy and Theory of Artificial Intelligence” that was held in October 2011 in Thessaloniki. Artificial Intelligence is perhaps unique among engineering subjects in that it has raised very basic questions about the nature of computing, perception, reasoning, learning, language, action, interaction, (...) consciousness, humankind, life etc. etc. – and at the same time it has contributed substantially to answering these questions. There is thus a substantial tradition of work, both on AI by philosophers and of theory within AI itself. - The volume contains papers by Bostrom, Dreyfus, Gomila, O'Regan and Shagrir. (shrink)