When, What, and How Much to Reward in Reinforcement Learning-Based Models of Cognition

Cognitive Science 36 (2):333-358 (2012)
Reinforcement learning approaches to cognitive modeling represent task acquisition as learning to choose the sequence of steps that accomplishes the task while maximizing a reward. However, an apparently unrecognized problem for modelers is choosing when, what, and how much to reward; that is, when (the moment: end of trial, subtask, or some other interval of task performance), what (the objective function: e.g., performance time or performance accuracy), and how much (the magnitude: with binary, categorical, or continuous values). In this article, we explore the problem space of these three parameters in the context of a task whose completion entails some combination of 36 state–action pairs, where all intermediate states (i.e., after the initial state and prior to the end state) represent progressive but partial completion of the task. Different choices produce profoundly different learning paths and outcomes, with the strongest effect for moment. Unfortunately, there is little discussion in the literature of the effect of such choices. This absence is disappointing, as the choice of when, what, and how much needs to be made by a modeler for every learning model
Keywords Adaptive behavior  Reinforcement learning  Strategy selection  Expected utility  Skill acquisition and learning  Choice  Expected value  Cognitive architecture
Categories (categorize this paper)
DOI 10.1111/j.1551-6709.2011.01222.x
 Save to my reading list
Follow the author(s)
My bibliography
Export citation
Find it on Scholar
Edit this record
Mark as duplicate
Revision history
Request removal from index
Download options
Our Archive

Upload a copy of this paper     Check publisher's policy     Papers currently archived: 27,678
Through your library
References found in this work BETA

View all 10 references / Add more references

Citations of this work BETA

Add more citations

Similar books and articles
The Evolution of Vagueness.Cailin O'Connor - 2013 - Erkenntnis (S4):1-21.
Reward: Wanted – a Better Definition.Irving Kupfermann - 2000 - Behavioral and Brain Sciences 23 (2):208-208.

Monthly downloads

Added to index


Total downloads

21 ( #237,064 of 2,170,072 )

Recent downloads (6 months)

2 ( #186,298 of 2,170,072 )

How can I increase my downloads?

My notes
Sign in to use this feature

There  are no threads in this forum
Nothing in this forum yet.

Other forums