Original ArticlesStatistical learning is constrained to less abstract patterns in complex sensory input (but not the least)
Introduction
Throughout our lifetime, experience shapes our mental model of the world; learning from patterns, regularities, or statistics in the environment is a way that experience might powerfully and incidentally shape cognition. Rich mental models or representations of the environment in turn support the prediction of upcoming sensory input facilitating both rapid and accurate cognitive processing when these predictions are correct, or error-signals and further learning and/or adaptation when these predictions are incorrect.
Indeed, statistical learning,1 or the sensitivity to statistical information in sensory input, is increasingly considered to form the foundation of numerous and diverse perceptual and cognitive abilities. For example, not only is statistical learning believed to be an essential component of the development of language (Romberg and Saffran, 2010, Saffran et al., 1996, Thiessen and Erickson, 2013), but researchers have increasingly turned to mechanisms of adaptation, or sensitivity to statistical or distributional patterns in the environment, to explain adult comprehension (e.g., Fine and Jaeger, 2013, Kleinschmidt and Jaeger, 2015). Likewise in vision, there have been many demonstrations that the visual system is sensitive to statistical information both immediately after exposure (e.g., neural demonstrations: Summerfield et al., 2008, Turk-Browne et al., 2009, Turk-Browne et al., 2010) and cumulatively over a lifetime of visual experience (Purves et al., 2014, Purves et al., 2011). Indeed, even young infants have the ability to perform complex visual statistical learning tasks (as young as 2-months-old for sequential visual statistical learning tasks; Fiser and Aslin, 2002, Kirkham et al., 2002 for spatial statistical learning tasks with 9-month-olds).2 The more the field searches, the more it finds that the sensory environment is filled with statistical information. Paired with the rapid and ubiquitous statistical learning abilities of both infants and adults in many different types of input, it is clear that understanding cognitive responses to statistical information is foundational to understanding many aspects of perception and cognition.
However, despite its clear importance, little is known of how statistical learning proceeds in our daily environments. A central problem is that it is unknown how statistical learning mechanism(s) operate given the richness of the statistical information that we routinely receive. In this paper, we focus on a particular type of rich information: what happens when statistical information is present across multiple levels of abstraction? How does learning proceed when an observer can learn that not only do categories of objects predict another, but also that individual objects within categories predict each other? This kind of overlapping statistical information is routinely presented to us. For example, the predictive relationship between dogs and leashes exists based on abstract categories as well as on the actual objects or exemplars seen in the world (e.g. dogs have their specific leashes). There are many other examples where both categories of objects and also the individual objects within these categories are predictive of each other: types of animals and their habitats (e.g., fish and bodies of water), chairs and tables, people and their personal affects (e.g., wallets, cell phones). Thus, when two groups of objects are paired in the world, there is visual statistical information from which one can learn, all the way from low-level visual similarities across exemplars within a category, to relationships between individual objects, to their semantic, abstract categories. While the current paper focuses on visual statistical learning, similar inputs are found in other domains; for example language input, where there is overlapping statistical information at the levels of phonology, lexical items, and syntactic information when one rehears a single utterance.
The current paper addresses what is learned when there is predictive visual information across levels of abstraction. Is statistical learning a priori constrained to learn particular patterns? There are two types of competing theories on this: The first proposes that statistical learning is largely unconstrained; the second proposes that statistical learning is inherently constrained to learn from particular types of regularities.
By and large, previous research has supported the view that learning mechanism(s) are largely unconstrained: Statistical learning has been demonstrated in multiple sensory modalities (e.g., Conway & Christiansen, 2005) and across a wide range of perceptual input. For example, in the visual modality, learning can occur from sequences of gestures (Baldwin, Anderrson, Saffran, & Myers, 2007) or abstract shapes (Fiser & Aslin, 2001) and from spatial and non-spatial information (Mayr, 1996). While the majority of these studies have focused on learning probabilistic relations between individual objects, there is evidence that learning can occur at higher levels of informational abstraction including based on categories of nonsense words (Reeder et al., 2013, Saffran, 2002) or familiar semantic categories (Brady & Oliva, 2008). Overall, these studies, among many others, have led to the belief that statistical learning is largely unconstrained. That is, if there is any reliable probabilistic information in the environment, humans can learn from it regardless of modality or level of abstraction.
If statistical learning were unconstrained, how would learning proceed when a learner is faced with statistical information across multiple levels of abstraction simultaneously? A logical prediction is that an observer would learn across these multiple levels of representation simultaneously and in parallel. While this topic has been largely unexamined, there is a handful of evidence that learning can interact across levels of representation: Onnis, Waterfall, and Edelman (2008) found that the narrative structure of successive utterances supported statistical learning of individual syllables in adult learners; similarly, Saffran (2001) found that syntactic information in the form of phrase structures aided learning at lower levels (i.e., learning across individual elements), and Koch and Hoffmann (2000) suggested that different levels of information can compete in a serial reaction time (SRT) task.
However, the evidence that statistical learning can occur for all types of input and all levels of abstraction has been obtained under very constrained experimental conditions which may not reveal how learning operates over the rich statistical input encountered “in the wild”. To illustrate, Brady and Oliva (2008) use a paradigm where the categories of scenes are predictive of picture order but individual scenes are not (e.g. beaches predict kitchens as categories of scenes but beach1 does not predict kitchen1). Participants show evidence of learning the relationship between abstract categories of scenes, but it is not possible to learn based on individual scenes because only category-level regularities are present. While studies such as Brady and Oliva (2008) provide essential existence proofs, statistical learning has not been investigated in the context of the rich input that an everyday learner encounters, where multiple types of statistical regularities are present simultaneously.
The alternative theory is that statistical learning does have some constraints and that these constraints bias what is learned when presented with rich input. These theoretical views come in two types: The first type directly concerns the question of domain-generality and asserts that learning is biased towards a particular type of input (e.g., auditory vs. visual; linguistic vs. non-linguistic).3 The question of domain-generality is not directly addressed in the current paper, as we do not compare learning across types of sensory input but learning across multiple levels of abstraction within largely the same sensory input. The second family of theories, directly addressed here, asserts that there is a bias in the types of statistical information that will be learned within a given input. Newport, 1990, Elman, 1993 suggest that with language learning “less is more” and learners “start small,” respectively. Both of these views suggest that the most tractable patterns, or those connecting smaller units in the input, will be learned first. A recently emerging view considers the relationship between prediction and statistical learning and proposes that the goal of statistical learning is to acquire the most complete model of the environment and to enable the best possible prediction of the upcoming stimuli (Emberson et al., 2013, Karuza et al., 2014, Lupyan, 2015). These views are grounded in the theory of the brain as a predictive system with the goal of minimizing prediction error (e.g., predictive coding; Clark, 2013, Friston, 2005; reinforcement learning, associative learning and/or conditioning: Rescorla, 1988, Schultz et al., 1997). These views both provide a clearer prediction of what would be learned when encountering rich statistical input: in the former case, that learning would be constrained to either the smallest available units or simplest patterns in the input; in the latter case, the learner would learn from the statistical information that would provide the best possible prediction of future sensory input.
The current paper examines learning when participants are exposed to statistical information at multiple levels of abstraction. Investigating how learning proceeds over statistically rich input will allow some dissociation between constrained and unconstrained views of statistical learning. Specifically, do participants learn from the multiple levels of predictive dependencies simultaneously, as would be predicted by unconstrained theories of statistical learning, or are learners biased to learn at a certain level of abstraction? Furthermore, if learners are constrained, do they learn from the smallest and most tractable patterns and/or do they acquire representations that will best allow them to best reduce their prediction error for upcoming sensory input?
To address these questions, we devised a novel statistical learning task where predictive regularities are learnable and redundant across two levels of abstraction and investigated these questions in the domain of visual statistical learning of familiar objects and semantic categories. For example, in a stream of passively viewed pictures, the semantic categories (dogs, fish, flowers, birds) as well as the individual exemplars of these categories (e.g. dog1, fish1, dog2, fish2) were predictive of picture order (see Fig. 1). In other words, there was learnable statistical information at both the category-level and the object-level of the familiarization stream. We examined whether participants learned from both the statistical information linking the semantic categories as well as the individual objects, or whether they were biased to learn at a particular level of abstraction and if so, which level they are biased to.
Section snippets
Experiment 1: establishing category-level learning with shuffled exemplars
Before examining learning in the presence of two levels of abstraction, we first sought to replicate and extend the finding from Brady and Oliva (2008): namely that there is category-level learning when individual objects are not predictive of each other. In other words, we sought to find evidence of learning at the level of abstract semantic categories when individual objects are not predictive of picture order. We extended the previous findings from Brady and Oliva (2008) by employing new
Experiment 2: testing for object-level learning
Building on Experiment 1, we examine whether participants continue to learn from statistical regularities between semantic categories when more concrete object-specific regularities are present. In a small change from the previous study, participants view streams of pictures where the individual objects or pictures, as well as the categories, are predictive of picture. To illustrate, while in Experiment 1 pictures of birds predicted pictures of dogs but no individual bird predicted any
Experiment 3: testing for additional category-level knowledge
Having established that participants can learn from object-specific statistical regularities in the presence of more abstract category-level statistical regularities, Experiment 3 examined whether learning occurs along multiple levels of abstraction simultaneously (i.e., objects: bird1-dog1; semantic categories: birds-dogs) or whether participants preferentially learn based on statistical regularities of objects as suggested by the lack of typicality effect in Experiment 2. To this end, we
Experiment 4: learning from snapshots of individual objects
The findings from the previous experiments suggest that participants learn preferentially based on statistical regularities of individual objects. Indeed, when object-specific regularities are present, participants appear to be biased towards learning these regularities over regularities of semantic categories. However, the evidence is ambiguous as to whether they are preferentially learning based on individual objects or whether participants are learning based on the most specific statistical
General discussion
Our daily sensory input – the information that any statistical learning mechanism(s) must use to learn about the structure of the surrounding environment – delivers much richer statistical information than the tasks that are typically employed in the laboratory. We investigated the nature of statistical learning when the statistical information being encountered is richer and more complex than in previous studies, and specifically, when there is learnable statistical information present at
Acknowledgments
We’d like to thank Drs. Dima Amso, Rick Dale, Jordan DeLong, David Field, Thomas Farmer, Gary Lupyan, Adele Goldberg, Elika Bergelson, Toben Mintz and three anonymous Reviewers for helpful conversations and/or comments on the manuscript. We’d also like to thank Esteban Buz and Dave Kleinschmidt for their (statistical) support. Thank you to Claire Schmidt, Andrew Webb, Joey Ciufo, Mary Marchetti, Haley Weaver and Camila Rivero for help with data collection and, in particular, we thank Dr.
References (54)
Learning and development in neural networks: the importance of starting small
Cognition
(1993)- et al.
Is statistical learning constrained by lower level perceptual organization?
Cognition
(2013) - et al.
Domain generality versus modality specificity: The paradox of statistical learning
Trends in Cognitive Science
(2015) - et al.
Pictures and names: Making the connection
Cognitive Psychology
(1984) - et al.
Invited review: Combining fMRI and behavioral measures to examine the process of human learning
Neurobiology of Learning and Memory
(2014) - et al.
The neural correlated of statistical learning in a word segmentation task: An fMRI study
Brain and Language: Special Issue
(2013) Maturational constraints on language learning
Cognitive Science
(1990)- et al.
Learn locally, act globally: Learning language from variation set cues
Cognition
(2008) - et al.
Implicit learning and statistical learning: One phenomenon, two approaches
Trends in Cognitive Sciences
(2006) - et al.
From shared contexts to syntactic categories: The role of distributional information in learning linguistic form-classes
Cognitive Psychology
(2013)
Basic objects in natural categories
Cognitive Psychology
Words in a sea of sounds: The output of infant statistical learning
Cognition
Constraints on statistical language learning
Journal of Memory and Language
Statistical learning: from acquiring specific items to forming general rules
Current Directions in Psychological Science
Segmenting dynamic human action via statistical structure
Cognition
Visual long-term memory has a massive storage capacity for object details
Proceedings of the National Academy of Sciences
Statistical learning using real-world scenes: Extracting categorical regularities without conscious intent
Psychological Science
pwr: Basic functions for power analysis. R package version 1.1.1
Whatever next? Predictive brains, situated agents, and the future of cognitive science
Behavioral and Brain Sciences
Modality-constrained statistical learning of tactile, visual, and auditory sequences
Journal of Experimental Psychology: Learning, Memory, & Cognition
Statistical learning within and between modalities: pitting abstract against stimulus-specific representations
Psychological Science
Graded motor responses in the time course of categorizing atypical exemplars
Memory and Cognition
FMRI activity patterns in human LOC carry information about object exemplars within categories
Journal of Cognitive Neuroscience
Learning to sample: Eye tracking and fMRI indices of changes in object perception
Journal of Cognitive Neuroscience
Timing is everything: Change in presentation rate have opposite effects on auditory and visual implicit statistical learning
Quarterly Journal of Experimental Psychology
Evidence for error-based implicit learning in adult language processing
Cognitive Science
Unsupervised statistical learning of higher-order spatial structures from visual scenes
Psychological Science
Cited by (18)
Superior learning in synesthetes: Consistent grapheme-color associations facilitate statistical learning
2019, CognitionCitation Excerpt :Importantly, the variable mappings and experience of SC+ synesthetes also did not harm learning (as compared with the control and auditory-only groups). This suggests that there is no real cost to having variable or less consistent multimodal information, although the boundaries of this could be explored much further and work investigating how multiple levels of information are learned in statistical learning studies is greatly needed (c.f., (Brady & Oliva, 2008; Emberson & Rubinstein, 2016; Turk-Browne, Isola, Scholl, & Treat, 2008). It could be that when multiple modalities indicate a statistical relationship, they sum to produce a stronger learning signal and boost learning.
Local Patterns to Global Architectures: Influences of Network Topology on Human Learning
2016, Trends in Cognitive SciencesCitation Excerpt :Despite its wide applicability, an important and growing area of research probes constraints on our distributional learning capacities (i.e., how might this learning mechanism break down in the context of multiple patterns [78,79]? At what levels of abstraction does it operate [80]? And how does learning interact with attentional processes [81,82]?).
The blowfish effect: children and adults use atypical exemplars to infer more narrow categories during word learning
2019, Journal of Child LanguageIntracranial Entrainment Reveals Statistical Learning across Levels of Abstraction
2023, Journal of Cognitive NeuroscienceConceptual Associations Generate Sensory Predictions
2023, Journal of Neuroscience