Levels of representation in the electrophysiology of speech perception

Abstract

Mapping from acoustic signals to lexical representations is a complex process mediated by a number of different levels of representation. This paper reviews properties of the phonetic and phonological levels, and hypotheses about how category structure is represented at each of these levels, and evaluates these hypotheses in light of relevant electrophysiological studies of phonetics and phonology. The paper examines evidence for two alternative views of how infant phonetic representations develop into adult representations, a structure-changing view and a structure-adding view, and suggests that each may be better suited to different kinds of phonetic categories. Electrophysiological results are beginning to provide information about phonological representations, but less is known about how the more abstract representations at this level could be coded in the brain.

Introduction

Speech perception involves a mapping from continuous acoustic waveforms onto the discrete phonological units used to store words in the mental lexicon. For example, when we hear the word cat, we map a complex and continuous pattern of vibration at the eardrum onto a phonological percept which has just three clearly distinct pieces: /k/, /æ/ and /t/. A great deal of evidence indicates that this mapping from sound to words is not a simple one-step mapping, but is instead mediated by a number of different levels of representation. This article reviews studies of how the brain supports the different levels of representation, with a focus on work using the electrophysiological measures electroencephalography (EEG) and magnetoencephalography (MEG), and how it relates to hypotheses derived from behavioral and theoretical research in phonetics and phonology.

EEG and MEG provide noninvasive measures of neuronal activity in the brain. Electrodes positioned on the scalp, or magnetic field sensors positioned close to the scalp, measure changes in scalp voltages or scalp magnetic fields, entirely passively and with millisecond resolution. These measures are direct, and provide excellent temporal resolution, but only modest localization information. Whereas the other papers in this special issue discuss findings involving a number of different brain regions, most of the existing electrophysiological findings about speech perception have focused on evoked responses occurring within 250 ms after a sound is presented, and generated in auditory cortex. Human auditory cortex is situated on the superior plane of the temporal lobe, that is, on the lower side of the Sylvian fissure. Human auditory cortex consists of a number of subareas, but most electrophysiological findings about speech perception do not reliably implicate specific subareas of auditory cortex.

A great deal of electrophysiological research on speech has focused on two evoked responses: the auditory N100 and the mismatch response (see Fig. 1). The auditory N100, and its magnetic counterpart M100, are often referred to as exogenous responses, meaning that they are evoked by any acoustic stimulus with a well-defined onset, regardless of the listener’s task or attentional state (Näätänen & Picton, 1987). However, the latency, amplitude and localization of the N100 vary reliably when certain acoustic and perceptual parameters are varied, and there are reports of task-related modulation of M100 (e.g., Poeppel et al., 1996).

The auditory mismatch paradigm has been the most productive paradigm in the electrophysiological study of speech, and has revealed evidence of a number of different levels of representation. When a sequence of identical sounds, known as standards, is interrupted by infrequent deviant sounds, the deviant sounds elicit a characteristic response component known as the Mismatch Negativity (MMN) or Magnetic Mismatch Field (MMF; Näätänen et al 1978, Näätänen and Winkler 1999. The mismatch response typically occurs 150–250 ms after the onset of the deviant sound, and when localization information is available, it typically implicates supratemporal auditory cortex Hari et al 1984, Scherg et al 1989, Sams et al 1991, Alho 1995.¹ It can only be seen by comparing responses to infrequent deviant sounds to responses to frequent standard sounds. It can be elicited even when the subject does not attend to the stimuli: studies in this paradigm often present sounds while subjects read a book or watch a silent movie. Hence, it is commonly used as a measure of preattentive processing. However, it is also elicited when subjects actively attend to the stimuli.

Importantly for research on speech, the amplitude of the mismatch response also tends to increase as the discriminability of the standard and deviant stimuli increases Sams et al 1985, Lang et al 1990, Aaltonen et al 1993, Tiitinen et al 1994. For this reason a number of studies have examined whether mismatch response amplitudes track the discriminability profiles established in behavioral phonetics research, or are affected by the categorial status of pairs of sounds. The focus of this paper is on evidence for different levels of representation and how they might be encoded in the brain.