The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents [Book Review]

Minds and Machines 22 (2):71-85 (2012)
Abstract
This paper discusses the relation between intelligence and motivation in artificial agents, developing and briefly arguing for two theses. The first, the orthogonality thesis, holds (with some caveats) that intelligence and final goals (purposes) are orthogonal axes along which possible artificial intellects can freely vary—more or less any level of intelligence could be combined with more or less any final goal. The second, the instrumental convergence thesis, holds that as long as they possess a sufficient level of intelligence, agents having any of a wide range of final goals will pursue similar intermediary goals because they have instrumental reasons to do so. In combination, the two theses help us understand the possible range of behavior of superintelligent agents, and they point to some potential dangers in building such an agent
Keywords Superintelligence  Artificial intelligence  AI  Goal  Instrumental reason  Intelligent agent
Categories (categorize this paper)
Options
 Save to my reading list
Follow the author(s)
My bibliography
Export citation
Find it on Scholar
Edit this record
Mark as duplicate
Revision history Request removal from index
 
Download options
PhilPapers Archive


Upload a copy of this paper     Check publisher's policy on self-archival     Papers currently archived: 11,399
External links
Setup an account with your affiliations in order to access resources via your University's proxy server
Configure custom proxy (use this if your affiliation does not provide a proxy)
Through your library
References found in this work BETA
Nick Bostrom (2003). Are We Living in a Computer Simulation? Philosophical Quarterly 53 (211):243 - 255.
Nick Bostrom (2006). What is a Singleton. Linguistic and Philosophical Investigations 5 (2):48-54.
David J. Chalmers (2010). The Singularity: A Philosophical Analysis. Journal of Consciousness Studies 17 (9-10):9 - 10.
Derek Parfit (2011). On What Matters. Oxford University Press.

View all 8 references

Citations of this work BETA
Similar books and articles
Ralph Wedgwood (2011). Instrumental Rationality. Oxford Studies in Metaethics 6:280-309.
Stephen Ellis (2008). The Varieties of Instrumental Rationality. Southern Journal of Philosophy 46 (2):199-220.
Markos Valaris (2012). Instrumental Rationality. European Journal of Philosophy 20 (4):443-462.
Michael Bratman (2009). Intention, Belief, and Instrumental Rationality. In David Sobel & Steven Wall (eds.), Reasons for Action. Cambridge University Press. 13--36.
Thomas Kelly (2007). Evidence and Normativity: Reply to Leite. Philosophy and Phenomenological Research 75 (2):465–474.
Thomas Kelly (2003). Epistemic Rationality as Instrumental Rationality: A Critique. Philosophy and Phenomenological Research 66 (3):612–640.
Michael Smith (2004). Instrumental Desires, Instrumental Rationality. Aristotelian Society Supplementary Volume 78 (1):93–109.
Edward Harcourt (2004). Instrumental Desires, Instrumental Rationality. Aristotelian Society Supplementary Volume 78 (1):111–129.
John Brunero (2005). Instrumental Rationality and Carroll's Tortoise. Ethical Theory and Moral Practice 8 (5):557 - 569.
Analytics

Monthly downloads

Added to index

2012-06-14

Total downloads

24 ( #73,525 of 1,102,971 )

Recent downloads (6 months)

8 ( #29,688 of 1,102,971 )

How can I increase my downloads?

My notes
Sign in to use this feature


Discussion
Start a new thread
Order:
There  are no threads in this forum
Nothing in this forum yet.