Elsevier

Cognition

Volume 153, August 2016, Pages 63-78
Cognition

Original Articles
Statistical learning is constrained to less abstract patterns in complex sensory input (but not the least)

https://doi.org/10.1016/j.cognition.2016.04.010Get rights and content

Highlights

  • Examined learning of statistical regularities at multiple levels of abstraction.

  • Learning at the most abstract level occurs only in the absence of other information.

  • Learning preferentially occurs based on objects not at other levels of abstraction.

Abstract

The influence of statistical information on behavior (either through learning or adaptation) is quickly becoming foundational to many domains of cognitive psychology and cognitive neuroscience, from language comprehension to visual development. We investigate a central problem impacting these diverse fields: when encountering input with rich statistical information, are there any constraints on learning? This paper examines learning outcomes when adult learners are given statistical information across multiple levels of abstraction simultaneously: from abstract, semantic categories of everyday objects to individual viewpoints on these objects. After revealing statistical learning of abstract, semantic categories with scrambled individual exemplars (Exp. 1), participants viewed pictures where the categories as well as the individual objects predicted picture order (e.g., bird1—dog1, bird2—dog2). Our findings suggest that participants preferentially encode the relationships between the individual objects, even in the presence of statistical regularities linking semantic categories (Exps. 2 and 3). In a final experiment we investigate whether learners are biased towards learning object-level regularities or simply construct the most detailed model given the data (and therefore best able to predict the specifics of the upcoming stimulus) by investigating whether participants preferentially learn from the statistical regularities linking individual snapshots of objects or the relationship between the objects themselves (e.g., bird_picture1—dog_picture1, bird_picture2—dog_picture2). We find that participants fail to learn the relationships between individual snapshots, suggesting a bias towards object-level statistical regularities as opposed to merely constructing the most complete model of the input. This work moves beyond the previous existence proofs that statistical learning is possible at both very high and very low levels of abstraction (categories vs. individual objects) and suggests that, at least with the current categories and type of learner, there are biases to pick up on statistical regularities between individual objects even when robust statistical information is present at other levels of abstraction. These findings speak directly to emerging theories about how systems supporting statistical learning and prediction operate in our structure-rich environments. Moreover, the theoretical implications of the current work across multiple domains of study is already clear: statistical learning cannot be assumed to be unconstrained even if statistical learning has previously been established at a given level of abstraction when that information is presented in isolation.

Introduction

Throughout our lifetime, experience shapes our mental model of the world; learning from patterns, regularities, or statistics in the environment is a way that experience might powerfully and incidentally shape cognition. Rich mental models or representations of the environment in turn support the prediction of upcoming sensory input facilitating both rapid and accurate cognitive processing when these predictions are correct, or error-signals and further learning and/or adaptation when these predictions are incorrect.

Indeed, statistical learning,1 or the sensitivity to statistical information in sensory input, is increasingly considered to form the foundation of numerous and diverse perceptual and cognitive abilities. For example, not only is statistical learning believed to be an essential component of the development of language (Romberg and Saffran, 2010, Saffran et al., 1996, Thiessen and Erickson, 2013), but researchers have increasingly turned to mechanisms of adaptation, or sensitivity to statistical or distributional patterns in the environment, to explain adult comprehension (e.g., Fine and Jaeger, 2013, Kleinschmidt and Jaeger, 2015). Likewise in vision, there have been many demonstrations that the visual system is sensitive to statistical information both immediately after exposure (e.g., neural demonstrations: Summerfield et al., 2008, Turk-Browne et al., 2009, Turk-Browne et al., 2010) and cumulatively over a lifetime of visual experience (Purves et al., 2014, Purves et al., 2011). Indeed, even young infants have the ability to perform complex visual statistical learning tasks (as young as 2-months-old for sequential visual statistical learning tasks; Fiser and Aslin, 2002, Kirkham et al., 2002 for spatial statistical learning tasks with 9-month-olds).2 The more the field searches, the more it finds that the sensory environment is filled with statistical information. Paired with the rapid and ubiquitous statistical learning abilities of both infants and adults in many different types of input, it is clear that understanding cognitive responses to statistical information is foundational to understanding many aspects of perception and cognition.

However, despite its clear importance, little is known of how statistical learning proceeds in our daily environments. A central problem is that it is unknown how statistical learning mechanism(s) operate given the richness of the statistical information that we routinely receive. In this paper, we focus on a particular type of rich information: what happens when statistical information is present across multiple levels of abstraction? How does learning proceed when an observer can learn that not only do categories of objects predict another, but also that individual objects within categories predict each other? This kind of overlapping statistical information is routinely presented to us. For example, the predictive relationship between dogs and leashes exists based on abstract categories as well as on the actual objects or exemplars seen in the world (e.g. dogs have their specific leashes). There are many other examples where both categories of objects and also the individual objects within these categories are predictive of each other: types of animals and their habitats (e.g., fish and bodies of water), chairs and tables, people and their personal affects (e.g., wallets, cell phones). Thus, when two groups of objects are paired in the world, there is visual statistical information from which one can learn, all the way from low-level visual similarities across exemplars within a category, to relationships between individual objects, to their semantic, abstract categories. While the current paper focuses on visual statistical learning, similar inputs are found in other domains; for example language input, where there is overlapping statistical information at the levels of phonology, lexical items, and syntactic information when one rehears a single utterance.

The current paper addresses what is learned when there is predictive visual information across levels of abstraction. Is statistical learning a priori constrained to learn particular patterns? There are two types of competing theories on this: The first proposes that statistical learning is largely unconstrained; the second proposes that statistical learning is inherently constrained to learn from particular types of regularities.

By and large, previous research has supported the view that learning mechanism(s) are largely unconstrained: Statistical learning has been demonstrated in multiple sensory modalities (e.g., Conway & Christiansen, 2005) and across a wide range of perceptual input. For example, in the visual modality, learning can occur from sequences of gestures (Baldwin, Anderrson, Saffran, & Myers, 2007) or abstract shapes (Fiser & Aslin, 2001) and from spatial and non-spatial information (Mayr, 1996). While the majority of these studies have focused on learning probabilistic relations between individual objects, there is evidence that learning can occur at higher levels of informational abstraction including based on categories of nonsense words (Reeder et al., 2013, Saffran, 2002) or familiar semantic categories (Brady & Oliva, 2008). Overall, these studies, among many others, have led to the belief that statistical learning is largely unconstrained. That is, if there is any reliable probabilistic information in the environment, humans can learn from it regardless of modality or level of abstraction.

If statistical learning were unconstrained, how would learning proceed when a learner is faced with statistical information across multiple levels of abstraction simultaneously? A logical prediction is that an observer would learn across these multiple levels of representation simultaneously and in parallel. While this topic has been largely unexamined, there is a handful of evidence that learning can interact across levels of representation: Onnis, Waterfall, and Edelman (2008) found that the narrative structure of successive utterances supported statistical learning of individual syllables in adult learners; similarly, Saffran (2001) found that syntactic information in the form of phrase structures aided learning at lower levels (i.e., learning across individual elements), and Koch and Hoffmann (2000) suggested that different levels of information can compete in a serial reaction time (SRT) task.

However, the evidence that statistical learning can occur for all types of input and all levels of abstraction has been obtained under very constrained experimental conditions which may not reveal how learning operates over the rich statistical input encountered “in the wild”. To illustrate, Brady and Oliva (2008) use a paradigm where the categories of scenes are predictive of picture order but individual scenes are not (e.g. beaches predict kitchens as categories of scenes but beach1 does not predict kitchen1). Participants show evidence of learning the relationship between abstract categories of scenes, but it is not possible to learn based on individual scenes because only category-level regularities are present. While studies such as Brady and Oliva (2008) provide essential existence proofs, statistical learning has not been investigated in the context of the rich input that an everyday learner encounters, where multiple types of statistical regularities are present simultaneously.

The alternative theory is that statistical learning does have some constraints and that these constraints bias what is learned when presented with rich input. These theoretical views come in two types: The first type directly concerns the question of domain-generality and asserts that learning is biased towards a particular type of input (e.g., auditory vs. visual; linguistic vs. non-linguistic).3 The question of domain-generality is not directly addressed in the current paper, as we do not compare learning across types of sensory input but learning across multiple levels of abstraction within largely the same sensory input. The second family of theories, directly addressed here, asserts that there is a bias in the types of statistical information that will be learned within a given input. Newport, 1990, Elman, 1993 suggest that with language learning “less is more” and learners “start small,” respectively. Both of these views suggest that the most tractable patterns, or those connecting smaller units in the input, will be learned first. A recently emerging view considers the relationship between prediction and statistical learning and proposes that the goal of statistical learning is to acquire the most complete model of the environment and to enable the best possible prediction of the upcoming stimuli (Emberson et al., 2013, Karuza et al., 2014, Lupyan, 2015). These views are grounded in the theory of the brain as a predictive system with the goal of minimizing prediction error (e.g., predictive coding; Clark, 2013, Friston, 2005; reinforcement learning, associative learning and/or conditioning: Rescorla, 1988, Schultz et al., 1997). These views both provide a clearer prediction of what would be learned when encountering rich statistical input: in the former case, that learning would be constrained to either the smallest available units or simplest patterns in the input; in the latter case, the learner would learn from the statistical information that would provide the best possible prediction of future sensory input.

The current paper examines learning when participants are exposed to statistical information at multiple levels of abstraction. Investigating how learning proceeds over statistically rich input will allow some dissociation between constrained and unconstrained views of statistical learning. Specifically, do participants learn from the multiple levels of predictive dependencies simultaneously, as would be predicted by unconstrained theories of statistical learning, or are learners biased to learn at a certain level of abstraction? Furthermore, if learners are constrained, do they learn from the smallest and most tractable patterns and/or do they acquire representations that will best allow them to best reduce their prediction error for upcoming sensory input?

To address these questions, we devised a novel statistical learning task where predictive regularities are learnable and redundant across two levels of abstraction and investigated these questions in the domain of visual statistical learning of familiar objects and semantic categories. For example, in a stream of passively viewed pictures, the semantic categories (dogs, fish, flowers, birds) as well as the individual exemplars of these categories (e.g. dog1, fish1, dog2, fish2) were predictive of picture order (see Fig. 1). In other words, there was learnable statistical information at both the category-level and the object-level of the familiarization stream. We examined whether participants learned from both the statistical information linking the semantic categories as well as the individual objects, or whether they were biased to learn at a particular level of abstraction and if so, which level they are biased to.

Section snippets

Experiment 1: establishing category-level learning with shuffled exemplars

Before examining learning in the presence of two levels of abstraction, we first sought to replicate and extend the finding from Brady and Oliva (2008): namely that there is category-level learning when individual objects are not predictive of each other. In other words, we sought to find evidence of learning at the level of abstract semantic categories when individual objects are not predictive of picture order. We extended the previous findings from Brady and Oliva (2008) by employing new

Experiment 2: testing for object-level learning

Building on Experiment 1, we examine whether participants continue to learn from statistical regularities between semantic categories when more concrete object-specific regularities are present. In a small change from the previous study, participants view streams of pictures where the individual objects or pictures, as well as the categories, are predictive of picture. To illustrate, while in Experiment 1 pictures of birds predicted pictures of dogs but no individual bird predicted any

Experiment 3: testing for additional category-level knowledge

Having established that participants can learn from object-specific statistical regularities in the presence of more abstract category-level statistical regularities, Experiment 3 examined whether learning occurs along multiple levels of abstraction simultaneously (i.e., objects: bird1-dog1; semantic categories: birds-dogs) or whether participants preferentially learn based on statistical regularities of objects as suggested by the lack of typicality effect in Experiment 2. To this end, we

Experiment 4: learning from snapshots of individual objects

The findings from the previous experiments suggest that participants learn preferentially based on statistical regularities of individual objects. Indeed, when object-specific regularities are present, participants appear to be biased towards learning these regularities over regularities of semantic categories. However, the evidence is ambiguous as to whether they are preferentially learning based on individual objects or whether participants are learning based on the most specific statistical

General discussion

Our daily sensory input – the information that any statistical learning mechanism(s) must use to learn about the structure of the surrounding environment – delivers much richer statistical information than the tasks that are typically employed in the laboratory. We investigated the nature of statistical learning when the statistical information being encountered is richer and more complex than in previous studies, and specifically, when there is learnable statistical information present at

Acknowledgments

We’d like to thank Drs. Dima Amso, Rick Dale, Jordan DeLong, David Field, Thomas Farmer, Gary Lupyan, Adele Goldberg, Elika Bergelson, Toben Mintz and three anonymous Reviewers for helpful conversations and/or comments on the manuscript. We’d also like to thank Esteban Buz and Dave Kleinschmidt for their (statistical) support. Thank you to Claire Schmidt, Andrew Webb, Joey Ciufo, Mary Marchetti, Haley Weaver and Camila Rivero for help with data collection and, in particular, we thank Dr.

References (54)

  • E. Rosch et al.

    Basic objects in natural categories

    Cognitive Psychology

    (1976)
  • J.R. Saffran

    Words in a sea of sounds: The output of infant statistical learning

    Cognition

    (2001)
  • J.R. Saffran

    Constraints on statistical language learning

    Journal of Memory and Language

    (2002)
  • R.N. Aslin et al.

    Statistical learning: from acquiring specific items to forming general rules

    Current Directions in Psychological Science

    (2012)
  • D. Baldwin et al.

    Segmenting dynamic human action via statistical structure

    Cognition

    (2007)
  • T.F. Brady et al.

    Visual long-term memory has a massive storage capacity for object details

    Proceedings of the National Academy of Sciences

    (2008)
  • T.F. Brady et al.

    Statistical learning using real-world scenes: Extracting categorical regularities without conscious intent

    Psychological Science

    (2008)
  • S. Champely

    pwr: Basic functions for power analysis. R package version 1.1.1

  • A. Clark

    Whatever next? Predictive brains, situated agents, and the future of cognitive science

    Behavioral and Brain Sciences

    (2013)
  • C. Conway et al.

    Modality-constrained statistical learning of tactile, visual, and auditory sequences

    Journal of Experimental Psychology: Learning, Memory, & Cognition

    (2005)
  • C. Conway et al.

    Statistical learning within and between modalities: pitting abstract against stimulus-specific representations

    Psychological Science

    (2006)
  • R. Dale et al.

    Graded motor responses in the time course of categorizing atypical exemplars

    Memory and Cognition

    (2007)
  • E. Eger et al.

    FMRI activity patterns in human LOC carry information about object exemplars within categories

    Journal of Cognitive Neuroscience

    (2008)
  • L.L. Emberson et al.

    Learning to sample: Eye tracking and fMRI indices of changes in object perception

    Journal of Cognitive Neuroscience

    (2012)
  • L.L. Emberson et al.

    Timing is everything: Change in presentation rate have opposite effects on auditory and visual implicit statistical learning

    Quarterly Journal of Experimental Psychology

    (2011)
  • A.B. Fine et al.

    Evidence for error-based implicit learning in adult language processing

    Cognitive Science

    (2013)
  • J. Fiser et al.

    Unsupervised statistical learning of higher-order spatial structures from visual scenes

    Psychological Science

    (2001)
  • Cited by (18)

    • Superior learning in synesthetes: Consistent grapheme-color associations facilitate statistical learning

      2019, Cognition
      Citation Excerpt :

      Importantly, the variable mappings and experience of SC+ synesthetes also did not harm learning (as compared with the control and auditory-only groups). This suggests that there is no real cost to having variable or less consistent multimodal information, although the boundaries of this could be explored much further and work investigating how multiple levels of information are learned in statistical learning studies is greatly needed (c.f., (Brady & Oliva, 2008; Emberson & Rubinstein, 2016; Turk-Browne, Isola, Scholl, & Treat, 2008). It could be that when multiple modalities indicate a statistical relationship, they sum to produce a stronger learning signal and boost learning.

    • Local Patterns to Global Architectures: Influences of Network Topology on Human Learning

      2016, Trends in Cognitive Sciences
      Citation Excerpt :

      Despite its wide applicability, an important and growing area of research probes constraints on our distributional learning capacities (i.e., how might this learning mechanism break down in the context of multiple patterns [78,79]? At what levels of abstraction does it operate [80]? And how does learning interact with attentional processes [81,82]?).

    View all citing articles on Scopus
    View full text