The Missing Link Between Memory and Reinforcement Learning

Frontiers in Psychology 11 (2020)
  Copy   BIBTEX

Abstract

Reinforcement learning systems usually assume that a value function is defined over all states that can immediately give the value of a particular state or action. These values are used by a selection mechanism to decide which action to take. In contrast, when humans and animals make decisions, they collect evidence for different alternatives over time and take action only when sufficient evidence has been accumulated. We have previously developed a model of memory processing that includes semantic, episodic and working memory in a comprehensive architecture. Here, we describe how this memory mechanism can support decision making when the alternatives cannot be evaluated based on immediate sensory information alone. Instead we first imagine, and then evaluate a possible future that will result from choosing one of the alternatives. Here we present an extended model that can be used as a model for decision making that depends on accumulating evidence over time, whether that information comes from the sequential attention to different sensory properties or from internal simulation of the consequences of making a particular choice. We show how the new model explains both simple immediate choices, choices that depend on multiple sensory factors and complicated selections between alternatives that require forward looking simulations based on episodic and semantic memory structures. In this framework, vicarious trial and error is explained as an internal simulation that accumulates evidence for a particular choice. We argue that a system like this forms the “missing link” between more traditional ideas of semantic and episodic memory, and the associative nature of reinforcement learning.

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 76,442

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Analysis on Mental Structures in Language Learning.Ya-Ping Cui - 2005 - Philosophy of the Social Sciences 35 (3):147-150.
Is memory preservation?Mohan Matthen - 2010 - Philosophical Studies 148 (1):3-14.
Is mental time travel real time travel?Michael Barkasi & Melanie G. Rosen - 2020 - Philosophy and the Mind Sciences 1 (1):1-27.
The making of a memory mechanism.Carl F. Craver - 2003 - Journal of the History of Biology 36 (1):153-95.
Memory and Consciousness.Paula Droege - 2013 - Philosophia Scientae 17:171-193.
Memory and Consciousness.Paula Droege - 2013 - Philosophia Scientiae 17 (2):171-193.

Analytics

Added to PP
2020-12-22

Downloads
14 (#733,752)

6 months
3 (#227,700)

Historical graph of downloads
How can I increase my downloads?

Author Profiles

Christian Balkenius
Lund University
Annika Wallin
Lund University
Peter Gärdenfors
Lund University

Citations of this work

No citations found.

Add more citations