Evidence for a single mechanism gating perceptual and long-term memory information into working memory

Abstract

An influential view of working memory (WM) holds that its contents are controlled by a selective gating mechanism that allows for relevant perceptual information to enter WM when opened, but shields WM contents from interference when closed. In support of this idea, prior studies using the reference-back paradigm have established behavioral costs for opening and closing the gate between perception and WM. WM also frequently requires input from long-term memory (LTM), but it is currently unknown whether a similar gate controls the selection of LTM representations into WM, and how WM gating of perceptual vs. LTM sources of information relate to each other. To address these key theoretical questions, we devised a novel version of the reference-back paradigm, where participants switched between gating perceptual and LTM information into WM. We observed clear evidence for gate opening and closing costs in both cases. Moreover, the pattern of costs associated with gating and input source-switching indicated that perceptual and LTM information is gated into WM via a single gate, and rely on a shared source-selection mechanism. These findings extend current models of WM gating to encompass LTM information, and outline a new functional WM architecture.

Introduction

Imagine you are teaching a class and need to bring to mind a student's name in order to call on her after an ongoing exercise is completed. You can either look at the class' name list (i.e., searching a perceptual source of information) or try to retrieve the name from memory (i.e., searching a mnemonic source of information). Once you found it, in either case you would need to first select and then maintain in mind the student's name, while discarding other students' names that you come across during the search process. Working memory (WM) is the cognitive system that allows us to select and temporarily maintain information in mind to guide ongoing behavior (Cowan, 2017; Oberauer, 2002, Oberauer, 2009). The above example illustrates two key characteristics of WM. First, the contents of WM are selected either from perceptual input or from long-term memory (LTM). Second, WM needs to grant selective access to task-relevant information, while remaining closed to potential intrusion from task-irrelevant information. Surprisingly, we currently lack an understanding of how the processes responsible for this type of selective updating of WM may differ and/or interact between input domains, that is, depending on whether information is selected from perception or LTM. The goal of the present study was to examine the nature and relationship of WM updating from perception versus memory.

Previous research on how WM meets the demands of selective updating has primarily focused on how perceptual information is managed. Arguably the most influential neuro-cognitive model of WM regulation is the prefrontal basal ganglia working memory (PBWM) model, which proposes the existence of a dynamic gate between WM and perception (Frank, Loughry, & O'Reilly, 2001; O'Reilly & Frank, 2006). WM content is maintained, or shielded from irrelevant perceptual information when the gate is closed, but updated when the gate is opened, allowing new information to enter. The model separates the actual updating (thought to consist of changing active representations in prefrontal cortex) from the process that enables it, i.e. gate opening (proposed to be carried out by the basal ganglia).

The PBWM model's assumptions are supported by research using the reference-back paradigm (see Methods section), which has demonstrated separable response time (RT) performance costs for updating, gate-opening, and gate-closing (Rac-Lubashevsky and Kessler, 2016a, Rac-Lubashevsky and Kessler, 2016b). In the reference-back task, participants are presented on each trial with either the letter O or X (see Fig. 1). Each letter is surrounded by either a red or blue colored frame. On the first trial of the task, the participant encodes the letter they see into WM, and the letters shown on subsequent trials have to be compared to that reference stimulus, requiring a same/different response. Crucially, for subsequent blue-frame trials (“comparison” trials), the referent remains the same. Participants compare the WM stimulus with the on-screen stimulus, but WM content does not need to be updated, thus requiring a closed WM gate. By contrast, for red-frame trials (“reference” trials), the referent in WM has to be replaced with the stimulus shown in the red frame, thus requiring the gate to WM to be opened. When taking into account the nature of the current trial (n) and the preceding trial (n − 1), gate opening and closing costs can be isolated. That is, the gate opening cost is revealed by contrasting reference trials preceded by a comparison trial to those preceded by a reference trial, i.e., moving from a maintenance to an updating mode vs. staying in an updating mode. Similarly, the gate closing cost can be calculated by contrasting a comparison trial preceded by a reference trial to one preceded by a comparison trial, i.e., moving from an updating to a maintenance mode vs. staying in a maintenance mode. Previous work has shown dissociable individual-differences patterns (Rac-Lubashevsky & Kessler, 2016b) and neural correlates (Nir-Cohen, Kessler, & Egner, 2020; Rac-Lubashevsky & Kessler, 2018) involving the abovementioned processes.

Importantly, however, in addition to regulating the access of external, perceptual information to WM, we often need to do the same with internal information, that is, representations that originate in LTM. On the one hand, ongoing task performance can potentially be disrupted by internal information, for example, in the form of spontaneous thoughts that are unhelpful to the task at hand, or mind wandering (for a review, see Seli, Risko, Smilek, & Schacter, 2016). To preempt or mitigate such interruptions, the access of LTM representations to WM would have to be prevented. On the other hand, internal representations stored in LTM often need to be actively selected as task-relevant inputs to WM. For example, we may need to bring to mind an image of our car keys to use as a search template when trying to find our keys on a cluttered table. In other words, similar to perception, a gating mechanism is required that controls the content that is allowed to enter into WM from LTM.

While selective WM input-gating is clearly a requirement for dealing with both perceptual and LTM representations, very little research has asked whether and how gating mechanisms might differ between the two sources of information. While the PBWM model has not considered multiple gate, given that (attention to) perception and LTM are at least in part subserved by distinct neural systems (e.g., Chun, Golomb, & Turk-Browne, 2011), it is plausible that separate WM gating mechanisms may apply to these sources. One previous attempt at addressing this question is a study by Roth and Courtney (2007), who conducted an fMRI experiment to investigate whether a single neural mechanism may be responsible for WM updating with perceptual vs. memory information. Participants memorized different pairs of pictures of houses before the start of the main task. During the main task, they judged whether a picture presented on screen matched the one currently maintained in WM. For some trials, however, no judgment was required. Instead, based on an auditory cue, they either updated WM with the picture presented on screen, updated WM with a memorized associate for the picture on screen, or maintained the current picture. The authors compared reaction time and error rates between both types of updating, but did not detect any differences. At the neural level, they found that the same fronto-parietal set of brain regions appeared to be involved in both types of updating, with additional activity in the precuneus observed during memory retrieval. This design, however, did not isolate the actual gate opening and closing processes that control WM content, as it did not allow for the relevant comparison to successive updating trials. As such, no conclusions about these gating processes and their underlying mechanism can be drawn. Nonetheless, this study provides preliminary evidence that a single mechanism might be responsible for WM updating with perceptual and LTM information.

Given the necessity of gating operations for both perceptual and LTM inputs to WM, a first, fundamental question addressed in the present study was whether we could detect gating costs for both sources of information. Gate opening and closing costs have been repeatedly observed for perceptual stimuli (Rac-Lubashevsky and Kessler, 2016a, Rac-Lubashevsky and Kessler, 2016b). However, there currently exists no evidence for these costs when LTM representations are gated into WM, so a first aim of this study was to detect whether these costs exist.

A second question of high theoretical importance is whether there is a single shared, or two separate gating mechanisms responsible for updating WM with information from perception vs. LTM (see Fig. 2A-B). At first blush, dual gates seem necessary. For example, when the gate to WM is opened for perceptual information, distracting information from LTM would potentially be able to enter in the case of a single gate. Alternatively, a single gate suffices if the correct source of information can be selected independently of the gating process itself. Evidence for such a source selection mechanism has recently been observed (Verschooren, Liefooghe, Brass, & Pourtois, 2019; Verschooren, Pourtois, & Egner, 2020; Verschooren, Schindler, De Raedt, & Pourtois, 2019). Verschooren, Liefooghe, et al. (2019) instructed participants to commit four non-verbalizable drawings to LTM before the start of the experiment and to perform a probe-to-target matching task where the target, on a trial-by-trial basis, was either a figure presented on screen (perception trial) or a cue referring to a previously memorized figure (memory trial). A reliable cost was observed for switching between perception and memory referents, suggesting an attentional selection mechanism that switches between relevant information sources (see also Burgess, Dumontheil, & Gilbert, 2007). This type of source selection could in principle also be carried out before information is gated into WM. Moreover, there is evidence indicating that merely attending an item is not sufficient for encoding that item in WM (see discussion in Oberauer, 2019); thus, encoding of information into WM requires not only attentional selection, but also an open gate to WM. This implies that attentional source selection can take place independently of gating operations, which is a key assumption for the predictions we spell out below. That is, given this assumption, the relationship between source switch costs and gating costs can in principle reveal whether there is a single or there are dual gates for regulating the inflow of perceptual and mnemonic information into WM.

To address these questions, we used a modified reference-back paradigm in which, orthogonal to the trial type (comparison vs. reference trials), the target stimulus was either presented on screen or retrieved from LTM (see Fig. 3C). That is, participants either repeated or switched between comparison and reference trials, using either stimuli presented on screen or representations retrieved from LTM. Not only did this allow us to test for differences in gating for perceptual and LTM information, but also more generally to investigate how gate opening and closing operations interact with attention switches between perception and LTM input sources. Using the additive factors logic (Sternberg, 1969), different conclusions concerning a single vs. dual gating mechanisms can be drawn from distinct possible data patterns. In particular, we here derive rival RT pattern predictions based on different WM architectures (shared or dual gates for perceptual vs. LTM inputs to WM, see Fig. 2A, B) and different assumptions regarding the relationship between gating processes and source switch processes (whether they must proceed in series or can be carried out in parallel, see Fig. 2C-F). Before detailing the key predictions distinguishing between these architectures, note that, in line with findings on the “standard” reference-back task, we expected participants to generally respond faster on comparison than on reference trials, but to show a larger gating cost (gate-closing) on comparison than on reference trials (gate-opening) (see Fig. 2C-F; e.g., Rac-Lubashevsky & Kessler, 2016a).

We first provide arguments for making distinct predictions for gate opening and closing, and subsequently present the different RT patterns. Specifically, there is an important a priori difference between gate opening and closing that motivates different predictions for them. Goal-directed gate opening, by definition, is a selective process, as it needs to select whether WM will be updated with the upcoming information or not (Frank et al., 2001; O'Reilly & Frank, 2006). The selection of the input source to WM, by definition, is also a selective process. Assuming that these selection processes are both attentional in nature leads to the prediction that source switches and gate opening are likely to operate serially. Gate closing, on the other hand, can be non-selective (i.e., a closed gate blocks all inputs), and could thus occur in parallel with attentional switches between perception and memory.

As source selection and gate opening operate serially, we would expect their costs to be additive (Fig. 2C, D). When we combine this serial processing assumption with a single-gate architecture (Fig. 2A), we would expect the costs of gate-opening and source-switching to add up when both processes have to be carried out on the same trial. When the input source is switched but the gating state is repeated (i.e., when reference trials are repeated), no additional cost for gating is expected: a single gate can remain open or closed, even when the relevant input source is switched, given the assumption of independence of source selection and gate state (Oberauer, 2019) (Fig. 2C). On the other hand, when we combine the serial processing assumption with a dual-gates architecture (Fig. 2B), the relevant gate to WM would need to be opened whenever the input source is switched. This would entail a gate opening cost when attention is switched between input sources, even when the gating operation is repeated (i.e., a repetition of a reference trial). As a result, we would expect increased RTs in this particular condition, similar to the one where attention is switched and a change in the gate state takes place (i.e., a switch towards a reference trials). That is, even though the gating and switching processes here are assumed to be independent and operating in series, the resulting RT pattern would not be additive (see Fig. 2D).

Gate closing is expected to proceed in parallel with source selection, in which case one would predict their costs to produce an underadditive interaction (e.g., Kessler, 2017; Souza, Oberauer, Gade, & Druey, 2012). Under this parallel-processing assumption, it is not possible to distinguish between a single vs. dual-gates architecture, as both would predict similar RT patterns (see Fig. 2E, F). That is, it would not be possible to determine whether an additional gating cost is present when the gating operation is repeated (i.e., a repetition of a comparison trial) but attention is switched between the input sources (forcing the closing of the irrelevant gate). Even though we predict that a cost associated with closing/opening the gate would still be present for a dual-gates architecture, it would coincide with the attention switch cost (see Fig. 2F), thus making the RT pattern indistinguishable from the one predicted under a single-gate architecture (see Fig. 2E).

To summarize, the modified reference-back paradigm developed here can be used to (i) determine whether gate opening and closing costs can be detected for both perceptual and LTM sources and to (ii) differentiate between a single vs. dual WM gates for perception and LTM. We derived these predictions by assuming (a) that previously demonstrated costs of gate opening and closing (e.g., Rac-Lubashevsky and Kessler, 2016a, Rac-Lubashevsky and Kessler, 2016b) and of source switching (Verschooren et al., 2020; Verschooren, Liefooghe, et al., 2019; Verschooren, Schindler, et al., 2019) will be evident in the present task, and (b) that attentional selection can be independent from gating (Oberauer, 2019). As both gating and source switching produce a cost, we can infer that this cost is hidden by some other factors (e.g. parallel processing) in situations when it is not observed directly. We here report a single experiment, but it represents a higher-powered replication of a pilot experiment that we report in the Supplementary Materials. The combined analysis, included in the Supplementary Materials as well, confirms the robustness of the results reported below.