The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents [Book Review]
David Bourget (Western Ontario)
David Chalmers (ANU, NYU)
Rafael De Clercq
Ezio Di Nucci
Jack Alan Reynolds
Learn more about PhilPapers
Minds and Machines 22 (2):71-85 (2012)
This paper discusses the relation between intelligence and motivation in artificial agents, developing and briefly arguing for two theses. The first, the orthogonality thesis, holds (with some caveats) that intelligence and final goals (purposes) are orthogonal axes along which possible artificial intellects can freely vary—more or less any level of intelligence could be combined with more or less any final goal. The second, the instrumental convergence thesis, holds that as long as they possess a sufficient level of intelligence, agents having any of a wide range of final goals will pursue similar intermediary goals because they have instrumental reasons to do so. In combination, the two theses help us understand the possible range of behavior of superintelligent agents, and they point to some potential dangers in building such an agent.
|Keywords||Superintelligence Artificial intelligence AI Goal Instrumental reason Intelligent agent|
|Categories||categorize this paper)|
Setup an account with your affiliations in order to access resources via your University's proxy server
Configure custom proxy (use this if your affiliation does not provide a proxy)
|Through your library|
References found in this work BETA
Derek Parfit (2011). On What Matters. Oxford University Press.
Derek Parfit (1984). Reasons and Persons. Oxford University Press.
Neil Sinhababu (2009). The Humean Theory of Motivation Reformulated and Defended. Philosophical Review 118 (4):465-500.
David J. Chalmers (2010). The Singularity: A Philosophical Analysis. Journal of Consciousness Studies 17 (9-10):9 - 10.
Michael Smith (1987). The Humean Theory of Motivation. Mind 96 (381):36-61.
Citations of this work BETA
John Danaher (2015). Why AI Doomsayers Are Like Sceptical Theists and Why It Matters. Minds and Machines 25 (3):231-246.
Stuart Armstrong, Anders Sandberg & Nick Bostrom (2012). Thinking Inside the Box: Controlling and Using an Oracle AI. Minds and Machines 22 (4):299-324.
Stuart Armstrong, Nick Bostrom & Carl Shulman (forthcoming). Racing to the Precipice: A Model of Artificial Intelligence Development. AI and Society.
Similar books and articles
Ralph Wedgwood (2011). Instrumental Rationality. Oxford Studies in Metaethics 6:280-309.
Danny Frederick, Adversus Homo Economicus: Critique of Lester’s Account of Instrumental Rationality.
Donald C. Hubin (1999). What's Special About Humeanism. Noûs 33 (1):30-45.
Stephen Ellis (2008). The Varieties of Instrumental Rationality. Southern Journal of Philosophy 46 (2):199-220.
Markos Valaris (2012). Instrumental Rationality. European Journal of Philosophy 20 (4):443-462.
Michael Bratman (2009). Intention, Belief, and Instrumental Rationality. In David Sobel & Steven Wall (eds.), Reasons for Action. Cambridge University Press 13--36.
Thomas Kelly (2007). Evidence and Normativity: Reply to Leite. Philosophy and Phenomenological Research 75 (2):465–474.
Bruno Verbeek (2001). Consequentialism, Rationality and the Relevant Description of Outcomes. Economics and Philosophy 17 (2):181-205.
Thomas Kelly (2003). Epistemic Rationality as Instrumental Rationality: A Critique. Philosophy and Phenomenological Research 66 (3):612–640.
Michael Smith (2004). Instrumental Desires, Instrumental Rationality. Aristotelian Society Supplementary Volume 78 (1):93–109.
Edward Harcourt (2004). Instrumental Desires, Instrumental Rationality. Aristotelian Society Supplementary Volume 78 (1):111–129.
John Brunero (2005). Instrumental Rationality and Carroll's Tortoise. Ethical Theory and Moral Practice 8 (5):557 - 569.
Mark van Roojen (1995). Humean Motivation and Humean Rationality. Philosophical Studies 79 (1):37-57.
Added to index2012-06-14
Total downloads58 ( #69,954 of 1,790,003 )
Recent downloads (6 months)18 ( #44,321 of 1,790,003 )
How can I increase my downloads?