Superintelligence and the Future of Governance: On Prioritizing the Control Problem at the End of History

In Roman Yampolskiy (ed.), Artificial Intelligence Safety and Security. CRC Press (forthcoming)

This chapter argues that dual-use emerging technologies are distributing unprecedented offensive capabilities to nonstate actors. To counteract this trend, some scholars have proposed that states become a little “less liberal” by implementing large-scale surveillance policies to monitor the actions of citizens. This is problematic, though, because the distribution of offensive capabilities is also undermining states’ capacity to enforce the rule of law. I will suggest that the only plausible escape from this conundrum, at least from our present vantage point, is the creation of a “supersingleton” run by a friendly superintelligence, founded upon a “post-singularity social contract.” In making this argument, the present chapter offers a novel reason for prioritizing the “control problem,” i.e., the problem of ensuring that a greaterthan-human-level AI will positively enhance human well-being.
Keywords artificial intellgience  superintelligence  governance  ethics  control problem
Categories (categorize this paper)
Edit this record
Mark as duplicate
Export citation
Find it on Scholar
Request removal from index
Revision history

Download options

Our Archive

Upload a copy of this paper     Check publisher's policy     Papers currently archived: 44,474
External links

Setup an account with your affiliations in order to access resources via your University's proxy server
Configure custom proxy (use this if your affiliation does not provide a proxy)
Through your library

References found in this work BETA

The Extended Mind.Andy Clark & David J. Chalmers - 1998 - Analysis 58 (1):7-19.

View all 21 references / Add more references

Citations of this work BETA

No citations found.

Add more citations

Similar books and articles

Superintelligence as Superethical.Steve Petersen - 2017 - In Patrick Lin, Keith Abney & Ryan Jenkins (eds.), Robot Ethics 2.0. New York, USA: Oxford University Press. pp. 322-337.
Leakproofing the Singularity Artificial Intelligence Confinement Problem.Roman Yampolskiy - 2012 - Journal of Consciousness Studies 19 (1-2):194-214.
Leakproofing the Singularity.Roman V. Yampolskiy - 2012 - Journal of Consciousness Studies 19 (1-2):194-214.
Superintelligence as a Cause or Cure for Risks of Astronomical Suffering.Kaj Sotala & Lukas Gloor - 2017 - Informatica: An International Journal of Computing and Informatics 41 (4):389-400.
How Long Before Superintelligence?Nick Bostrom - 1998 - International Journal of Futures Studies 2.
Pharmaceutical Knowledge Governance: A Human Rights Perspective.Trudo Lemmens - 2013 - Journal of Law, Medicine and Ethics 41 (1):163-184.


Added to PP index

Total views
81 ( #103,183 of 2,273,208 )

Recent downloads (6 months)
25 ( #34,442 of 2,273,208 )

How can I increase my downloads?


My notes

Sign in to use this feature