The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents [Book Review]
David Bourget (Western Ontario)
David Chalmers (ANU, NYU)
Rafael De Clercq
Jack Alan Reynolds
Learn more about PhilPapers
Minds and Machines 22 (2):71-85 (2012)
This paper discusses the relation between intelligence and motivation in artificial agents, developing and briefly arguing for two theses. The first, the orthogonality thesis, holds (with some caveats) that intelligence and final goals (purposes) are orthogonal axes along which possible artificial intellects can freely vary—more or less any level of intelligence could be combined with more or less any final goal. The second, the instrumental convergence thesis, holds that as long as they possess a sufficient level of intelligence, agents having any of a wide range of final goals will pursue similar intermediary goals because they have instrumental reasons to do so. In combination, the two theses help us understand the possible range of behavior of superintelligent agents, and they point to some potential dangers in building such an agent
|Keywords||Superintelligence Artificial intelligence AI Goal Instrumental reason Intelligent agent|
|Categories||categorize this paper)|
Setup an account with your affiliations in order to access resources via your University's proxy server
Configure custom proxy (use this if your affiliation does not provide a proxy)
|Through your library|
References found in this work BETA
Nick Bostrom (2003). Are We Living in a Computer Simulation? Philosophical Quarterly 53 (211):243 - 255.
Nick Bostrom (2006). What is a Singleton. Linguistic and Philosophical Investigations 5 (2):48-54.
David J. Chalmers (2010). The Singularity: A Philosophical Analysis. Journal of Consciousness Studies 17 (9-10):9 - 10.
David Lewis (1988). Desire as Belief. Mind 97 (418):323-32.
Derek Parfit (2011). On What Matters. Oxford University Press.
Citations of this work BETA
Stuart Armstrong, Anders Sandberg & Nick Bostrom (2012). Thinking Inside the Box: Controlling and Using an Oracle AI. [REVIEW] Minds and Machines 22 (4):299-324.
Similar books and articles
Ralph Wedgwood (2011). Instrumental Rationality. Oxford Studies in Metaethics 6:280-309.
Danny Frederick, Adversus Homo Economicus: Critique of Lester’s Account of Instrumental Rationality.
Donald C. Hubin (1999). What's Special About Humeanism. Noûs 33 (1):30-45.
Stephen Ellis (2008). The Varieties of Instrumental Rationality. Southern Journal of Philosophy 46 (2):199-220.
Markos Valaris (2012). Instrumental Rationality. European Journal of Philosophy 20 (4):443-462.
Michael Bratman (2009). Intention, Belief, and Instrumental Rationality. In David Sobel & Steven Wall (eds.), Reasons for Action. Cambridge University Press. 13--36.
Thomas Kelly (2007). Evidence and Normativity: Reply to Leite. Philosophy and Phenomenological Research 75 (2):465–474.
Bruno Verbeek (2001). Consequentialism, Rationality and the Relevant Description of Outcomes. Economics and Philosophy 17 (2):181-205.
Thomas Kelly (2003). Epistemic Rationality as Instrumental Rationality: A Critique. Philosophy and Phenomenological Research 66 (3):612–640.
Michael Smith (2004). Instrumental Desires, Instrumental Rationality. Aristotelian Society Supplementary Volume 78 (1):93–109.
Edward Harcourt (2004). Instrumental Desires, Instrumental Rationality. Aristotelian Society Supplementary Volume 78 (1):111–129.
John Brunero (2005). Instrumental Rationality and Carroll's Tortoise. Ethical Theory and Moral Practice 8 (5):557 - 569.
Mark van Roojen (1995). Humean Motivation and Humean Rationality. Philosophical Studies 79 (1):37-57.
Added to index2012-06-14
Total downloads27 ( #68,741 of 1,101,637 )
Recent downloads (6 months)11 ( #21,823 of 1,101,637 )
How can I increase my downloads?