When, What, and How Much to Reward in Reinforcement Learning-Based Models of Cognition

Cognitive Science 36 (2):333-358 (2012)
Reinforcement learning approaches to cognitive modeling represent task acquisition as learning to choose the sequence of steps that accomplishes the task while maximizing a reward. However, an apparently unrecognized problem for modelers is choosing when, what, and how much to reward; that is, when (the moment: end of trial, subtask, or some other interval of task performance), what (the objective function: e.g., performance time or performance accuracy), and how much (the magnitude: with binary, categorical, or continuous values). In this article, we explore the problem space of these three parameters in the context of a task whose completion entails some combination of 36 state–action pairs, where all intermediate states (i.e., after the initial state and prior to the end state) represent progressive but partial completion of the task. Different choices produce profoundly different learning paths and outcomes, with the strongest effect for moment. Unfortunately, there is little discussion in the literature of the effect of such choices. This absence is disappointing, as the choice of when, what, and how much needs to be made by a modeler for every learning model
Keywords Adaptive behavior  Reinforcement learning  Strategy selection  Expected utility  Skill acquisition and learning  Choice  Expected value  Cognitive architecture
Categories (categorize this paper)
DOI 10.1111/j.1551-6709.2011.01222.x
 Save to my reading list
Follow the author(s)
My bibliography
Export citation
Find it on Scholar
Edit this record
Mark as duplicate
Revision history Request removal from index
Download options
PhilPapers Archive

Upload a copy of this paper     Check publisher's policy on self-archival     Papers currently archived: 22,631
External links
Setup an account with your affiliations in order to access resources via your University's proxy server
Configure custom proxy (use this if your affiliation does not provide a proxy)
Through your library
References found in this work BETA

View all 10 references / Add more references

Citations of this work BETA

Add more citations

Similar books and articles
Irving Kupfermann (2000). Reward: Wanted – a Better Definition. Behavioral and Brain Sciences 23 (2):208-208.

Monthly downloads

Added to index


Total downloads

18 ( #225,877 of 1,938,720 )

Recent downloads (6 months)

2 ( #290,621 of 1,938,720 )

How can I increase my downloads?

My notes
Sign in to use this feature

Start a new thread
There  are no threads in this forum
Nothing in this forum yet.