The ethical issues related to the possible future creation of machines with general intellectual capabilities far outstripping those of humans are quite distinct from any ethical problems arising in current automation and information systems. Such superintelligence would not be just another technological development; it would be the most important invention ever made, and would lead to explosive progress in all scientific and technological fields, as the superintelligence would conduct research with superhuman efficiency. To the extent that ethics is a cognitive pursuit, a superintelligence could also easily surpass humans in the quality of its moral thinking. However, it would be up to the designers of the superintelligence to specify its original motivations. Since the superintelligence may become unstoppably powerful because of its intellectual superiority and the technologies it could develop, it is crucial that it be provided with human-friendly motivations. This paper surveys some of the unique ethical issues in creating superintelligence, and discusses what motivations we ought to give a superintelligence, and introduces some cost-benefit considerations relating to whether the development of superintelligent machines ought to be accelerated or retarded.
|Keywords||No keywords specified (fix it)|
|Categories||categorize this paper)|
References found in this work BETA
No references found.
Citations of this work BETA
Autonomous Machines, Moral Judgment, and Acting for the Right Reasons.Duncan Purves, Ryan Jenkins & Bradley J. Strawser - 2015 - Ethical Theory and Moral Practice 18 (4):851-872.
Thinking Inside the Box: Controlling and Using an Oracle AI.Stuart Armstrong, Anders Sandberg & Nick Bostrom - 2012 - Minds and Machines 22 (4):299-324.
Self-Improving AI: An Analysis. [REVIEW]John Storrs Hall - 2007 - Minds and Machines 17 (3):249-259.
Similar books and articles
A Network Approach for Distinguishing Ethical Issues in Research and Development.Sjoerd D. Zwart, Ibo van de Poel, Harald van Mil & Michiel Brumsen - 2006 - Science and Engineering Ethics 12 (4):663-684.
Ethical Ideology and Ethical Judgment Regarding Ethical Issues in Business.Tim Barnett, Ken Bass & Gene Brown - 1994 - Journal of Business Ethics 13 (6):469 - 480.
Natural Problems and Artificial Intelligence.Tracy B. Henley - 1990 - Behavior and Philosophy 18 (2):43-55.
What's Different, Ethically, About Nanotechnology?: Foundational Questions and Answers. [REVIEW]Robert E. McGinn - 2010 - NanoEthics 4 (2):115-128.
Suggested Management Responses to Ethical Issues Raised by Technological Change.William P. Cordeiro - 1997 - Journal of Business Ethics 16 (12-13):1393-1400.
Added to index2009-01-28
Total downloads241 ( #14,773 of 2,169,122 )
Recent downloads (6 months)46 ( #5,282 of 2,169,122 )
How can I increase my downloads?