In Patrick Lin, Keith Abney & Ryan Jenkins (eds.), Robot Ethics 2.0. New York, USA: Oxford University Press. pp. 322-337 (2017)

Authors
Steve Petersen
Niagara University
Abstract
Nick Bostrom's book *Superintelligence* outlines a frightening but realistic scenario for human extinction: true artificial intelligence is likely to bootstrap itself into superintelligence, and thereby become ideally effective at achieving its goals. Human-friendly goals seem too abstract to be pre-programmed with any confidence, and if those goals are *not* explicitly favorable toward humans, the superintelligence will extinguish us---not through any malice, but simply because it will want our resources for its own purposes. In response I argue that things might not be as bad as Bostrom suggests. If the superintelligence must *learn* complex final goals, then this means such a superintelligence must in effect *reason* about its own goals. And because it will be especially clear to a superintelligence that there are no sharp lines between one agent's goals and another's, that reasoning could therefore automatically be ethical in nature.
Keywords superintelligence  ethics of artificial intelligence  value learning
Categories (categorize this paper)
Buy the book Find it on Amazon.com
Options
Edit this record
Mark as duplicate
Export citation
Find it on Scholar
Request removal from index
Revision history

Download options

PhilArchive copy

 PhilArchive page | Other versions
External links

Setup an account with your affiliations in order to access resources via your University's proxy server
Configure custom proxy (use this if your affiliation does not provide a proxy)
Through your library

References found in this work BETA

Reasons and Persons.Derek Parfit - 1984 - Oxford University Press.
The Sources of Normativity.Christine M. Korsgaard - 1996 - Cambridge University Press.
Superintelligence: Paths, Dangers, Strategies.Nick Bostrom (ed.) - 2014 - Oxford University Press.
The Possibility of Altruism.Thomas Nagel - 1970 - Oxford Clarendon Press.
Reasons and Persons.Joseph Margolis - 1986 - Philosophy and Phenomenological Research 47 (2):311-327.

View all 30 references / Add more references

Citations of this work BETA

Fully Autonomous AI.Wolfhart Totschnig - 2020 - Science and Engineering Ethics 26 (5):2473-2485.
Is It Good for Them Too? Ethical Concern for the Sexbots.Steve Petersen - 2017 - In John Danaher & Neil McArthur (eds.), Robot Sex: Social Implications and Ethical. Cambridge, USA: MIT Press. pp. 155-171.
Computational Goals, Values and Decision-Making.Louise A. Dennis - 2020 - Science and Engineering Ethics 26 (5):2487-2495.

Add more citations

Similar books and articles

How Long Before Superintelligence?Nick Bostrom - 1998 - International Journal of Futures Studies 2.
Superintelligence: Paths, Dangers, Strategies.Nick Bostrom (ed.) - 2014 - Oxford University Press.
Superintelligence as a Cause or Cure for Risks of Astronomical Suffering.Kaj Sotala & Lukas Gloor - 2017 - Informatica: An International Journal of Computing and Informatics 41 (4):389-400.

Analytics

Added to PP index
2018-02-12

Total views
798 ( #7,288 of 2,448,082 )

Recent downloads (6 months)
68 ( #9,146 of 2,448,082 )

How can I increase my downloads?

Downloads

My notes