Cognitive Science 36 (2):333-358 (2012)
Reinforcement learning approaches to cognitive modeling represent task acquisition as learning to choose the sequence of steps that accomplishes the task while maximizing a reward. However, an apparently unrecognized problem for modelers is choosing when, what, and how much to reward; that is, when (the moment: end of trial, subtask, or some other interval of task performance), what (the objective function: e.g., performance time or performance accuracy), and how much (the magnitude: with binary, categorical, or continuous values). In this article, we explore the problem space of these three parameters in the context of a task whose completion entails some combination of 36 state–action pairs, where all intermediate states (i.e., after the initial state and prior to the end state) represent progressive but partial completion of the task. Different choices produce profoundly different learning paths and outcomes, with the strongest effect for moment. Unfortunately, there is little discussion in the literature of the effect of such choices. This absence is disappointing, as the choice of when, what, and how much needs to be made by a modeler for every learning model
|Keywords||Adaptive behavior Reinforcement learning Strategy selection Expected utility Skill acquisition and learning Choice Expected value Cognitive architecture|
|Categories||categorize this paper)|
References found in this work BETA
How Can the Human Mind Occur in the Physical Universe?R. Anderson John - 2007 - Oup Usa.
The Information Capacity of the Human Motor System in Controlling the Amplitude of Movement.Paul M. Fitts - 1954 - Journal of Experimental Psychology 47 (6):381.
Deictic Codes for the Embodiment of Cognition.Dana H. Ballard, Mary M. Hayhoe, Polly K. Pook & Rajesh P. N. Rao - 1997 - Behavioral and Brain Sciences 20 (4):723-742.
Hierarchically Organized Behavior and its Neural Foundations: A Reinforcement Learning Perspective.Matthew M. Botvinick, Yael Niv & Andrew C. Barto - 2009 - Cognition 113 (3):262-280.
Soft Constraints in Interactive Behavior: The Case of Ignoring Perfect Knowledge in‐the‐World for Imperfect Knowledge in‐the‐Head*,*.Wayne D. Gray & Wai‐Tat Fu - 2004 - Cognitive Science 28 (3):359-382.
Citations of this work BETA
Interrogating Feature Learning Models to Discover Insights Into the Development of Human Expertise in a Real‐Time, Dynamic Decision‐Making Task.Sibert Catherine, D. Gray Wayne & K. Lindstedt John - 2017 - Topics in Cognitive Science 9 (2):374-394.
Interrogating Feature Learning Models to Discover Insights Into the Development of Human Expertise in a Real‐Time, Dynamic Decision‐Making Task.Catherine Sibert, Wayne D. Gray & John K. Lindstedt - 2016 - Topics in Cognitive Science 8 (4).
Predicting Short‐Term Remembering as Boundedly Optimal Strategy Choice.Andrew Howes, Geoffrey B. Duggan, Kiran Kalidindi, Yuan‐Chi Tseng & Richard L. Lewis - 2015 - Cognitive Science 40 (1).
Similar books and articles
Goal‐Proximity Decision‐Making.Vladislav D. Veksler, Wayne D. Gray & Michael J. Schoelles - 2013 - Cognitive Science 37 (4):757-774.
Are People Successful at Learning Sequences of Actions on a Perceptual Matching Task?Reiko Yakushijin & Robert A. Jacobs - 2011 - Cognitive Science 35 (5):939-962.
Positive Transfer and Negative Transfer/Anti-Learning of Problem Solving Skills.Magda Osman - unknown
Applying Forward Models to Sequence Learning: A Connectionist Implementation.Axel Cleeremans - unknown
Reward Prediction Error Signals Are Meta‐Representational.Nicholas Shea - 2012 - Noûs 48 (2):314-341.
Reward: Wanted – a Better Definition.Irving Kupfermann - 2000 - Behavioral and Brain Sciences 23 (2):208-208.
Using Regulatory Focus to Explore Implicit and Explicit Processing in Concept Learning.Arthur Markman, W. Maddox & G. C. Baldwin - 2007 - Journal of Consciousness Studies 14 (s 9-10):132-155.
Heterogeneity in Choices on Iowa Gambling Task: Preference for Infrequent–High Magnitude Punishment. [REVIEW]Varsha Singh & Azizuddin Khan - 2009 - Mind and Society 8 (1):43-57.
Added to index2012-01-19
Total downloads21 ( #237,064 of 2,170,072 )
Recent downloads (6 months)2 ( #186,298 of 2,170,072 )
How can I increase my downloads?