Concurrent processing of words and their replacements during speech

Abstract

Two picture naming experiments, in which an initial picture was occasionally replaced with another (target) picture, were conducted to study the temporal coordination of abandoning one word and resuming with another word in speech production. In Experiment 1, participants abandoned saying the initial name, and resumed with the name of the target picture. This triggered both interrupted (e.g., Mush- …scooter) and completed (mushroom …scooter) productions of the initial name. We found that the time from beginning naming the initial picture to ending it was longer when the target picture was visually degraded than when it was intact. In Experiment 2, participants abandoned saying the initial name, but without resuming. There was no visual degradation effect, and thus the effect did not seem to be driven by detection of the stopping cue. These findings demonstrate that planning a new word can begin before the initial word is abandoned, so that both words can be processed concurrently.

Introduction

People often need to undo one action and replace it with a different action. They do so when they realize they have just made or are making a mistake or when the environment changes in a way that renders the initial action unsatisfactory. In psycholinguistic research, the former case has been studied extensively (e.g., Levelt, 1983). It occurs when speakers realize that their utterance is erroneous or insufficiently informative to their addressees. But the latter case is also very common and occurs when the event that is being described changes in a relevant respect. For instance, a sportscaster might start saying that Horse A is leading (and this is true when he or she starts planning the utterance), but abandon the utterance when Horse B overtakes Horse A.

This article contrasts two accounts of the temporal coordination of abandoning a word and resuming with its replacement, in situations when the context changes. On one account, the resumption is planned after the abandoned word is stopped (Levelt, 1989). Alternatively, planning the resumption might begin when the abandoned word is still being uttered (Hartsuiker & Kolk, 2001), so that for a while the words are processed concurrently.

To detect the need to abandon utterances, speakers must monitor their own speech (e.g., Blackmer and Mitton, 1991, Hartsuiker and Kolk, 2001, Hartsuiker et al., 2005, Levelt, 1983, Levelt, 1989, Postma, 2000). They can monitor both overtly produced speech (external-channel monitoring) and the speech plan before articulation (internal-channel monitoring). The self-monitoring system inspects speech for linguistic well-formedness, but also for appropriateness in the context (e.g., Is the utterance specific enough for the interlocutor to discriminate among potential referents in the context? Is it still sensible given that some new event has just occurred?) and intelligibility to the interlocutor (Levelt, 1989). After detecting trouble, the monitor can give the signal to abandon ongoing speech and plan a resumption (see Hartsuiker & Kolk, 2001).

When the monitor gives this signal, speech does not stop immediately; stopping any action takes time (Logan & Cowan, 1984). This time-to-stop is estimated at about 150–200 ms (Hartsuiker and Kolk, 2001, Levelt, 1989, Slevc and Ferreira, 2006). According to Levelt’s theory of monitoring, the signal halts every component of the language production system at roughly the same time. A subsequent editing phase (often filled with uh or um) is used for the planning of the resumption. Thus, the production of the abandoned word and planning the resumption are sequential processes. In contrast, Hartsuiker and Kolk’s computational model of self-monitoring assumes that the stop signal and the planning of the resumption both begin immediately on the detection of trouble. Thus the production of the abandoned word and the planning of the resumption occur concurrently, from when the trouble is detected to when speech is stopped. This article attempts to arbitrate between these sequential and concurrent accounts.

Evidence against a sequential account comes from studies that considered the time from the interruption of speech to its resumption (i.e., interval 2–3 in Example 1).

• ₁Mush₂-, ₃Chair

In a corpus of naturalistic speech (obtained from a radio talk show), Blackmer and Mitton (1991) found that the interruption-to-resumption interval was less than 100 ms in almost 50% of their data, and was sometimes 0 ms. Oomen and Postma (2001) reported similar results in a picture naming experiment that elicited interruptions and resumptions. These results are difficult to reconcile with the sequential account, because the resumption would be planned during the interruption-to-resumption interval. It is unlikely that speakers can plan a word in less than 100 ms (e.g., it takes at least 170 ms to produce a prepared syllable; see Hartsuiker & Kolk, 2001).

In contrast, Hartsuiker and Kolk’s (2001) concurrent account can accommodate these findings, because the resumption has been wholly or partly prepared when speech is halted. In their simulations, model-generated distributions of interruption-to-resumption intervals corresponded well with data from Oomen and Postma (2001).

However, Hartsuiker and Kolk (2001) considered only speech errors and their repairs and not other cases where one needs to replace one word with another. It is possible that when an error occurs, planning processes are disordered, and hence error data are unrepresentative of error-free speech. For example, planning two words concurrently might even cause speech errors. Similarly, the observation of extremely short intervals between interruption and resumption in Blackmer and Mitton’s (1991) study appeared to be restricted to errors; for other types of replacements (e.g., when a word was insufficiently appropriate) much longer times were observed (i.e., 200–300 ms longer on average). Finally, note that Blackmer and Mitton’s and Oomen and Postma’s (2001) data on the time-course of interruption and resumption come from studies in which the experimenters had no control over each interruption and resumption (e.g., about the moment the speaker realized the need to resume).

Therefore, this study tests a further prediction of the concurrent account. If continued production of the to-be-abandoned word and planning of its replacement take place concurrently, then the resumption planning could influence the production of the abandoned word. In particular, an increase in the difficulty of resumption planning may increase the time to interrupt or complete the abandoned word. This is because two concurrent tasks may compete for a limited set of cognitive resources so that an increase in difficulty of one task slows down performance on the other task (e.g., Ferreira and Pashler, 2002, Meyer and Kieras, 1997, Pashler, 1994). In contrast, the sequential model does not predict any influence from the resumption on the duration of the abandoned word.

To test these predictions, we conducted an experiment that elicited abandoned words and resumptions (Experiment 1) and a control experiment that elicited abandoned words, but not resumptions (Experiment 2). Experiment 1 used a replanning paradigm in which speakers begin naming one picture (the initial picture), but that picture is quickly replaced by another picture (the target picture) on a small proportion of trials (following Hartsuiker et al., 2005). Our goal was to make speakers abandon their response to the initial picture and name the target picture instead. The majority of trials (87.5%) were no-change trials, so that participants would not anticipate the change on the change trials. In the change trials, the SOA was −300 ms (i.e., initial picture occurred 300 ms before target picture). This value was chosen so that participants would often not be able to stop themselves from saying at least part of the initial picture’s name.

Hartsuiker et al. (2005) manipulated the semantic and phonological relatedness between initial name and the target name. In all conditions, responses were observed in which participants interrupted within the initial name (Example 1), and responses in which the initial name was said completely (Example 2).

• ₁Mushroom₂, ₃Chair

The time to initiate the resumption (i.e., the time from target picture presentation to the point marked “3” in the examples) was affected by both semantic and phonological relatedness to the initial name, showing that the planning of resumptions is affected by their relationship to the abandoned word. Thus, Hartsuiker et al. demonstrated an influence of the abandoned word on resumption production.

The present study tested whether the resumption can influence the production of the abandoned word. We manipulated whether the target picture on change trials was easy or hard to describe. To do this, we presented pictures that either had intact contours or were visually degraded by deleting 50% of the contours. Naming degraded pictures takes longer than naming intact pictures (Meyer, Sleiderink, & Levelt, 1998). According to a sequential model, the resumption is planned only when speech stops (whether within a word or at its end). Thus, the time from initial word onset to interruption (i.e., 1–2 in Examples 1 and 2) should be unaffected by whether the target pictures were visually degraded (making resumption comparatively difficult) or not (making resumption comparatively easy).

In contrast, the concurrent model predicts that planning of the resumption begins as soon as the monitor detects the change in picture. Because planning the resumption requires processing resources, it should interfere with the concurrent production of the initial word. This prediction is independent of whether the initial word is interrupted (Example 1) or not (Example 2), because the model assumes that the resumption starts from change detection, which occurs before the actual interruption and also before the end of initial word completion (where the interruption process was presumably too slow to stop the word). Therefore, this model predicts that the onset-to-interruption interval (i.e., 1–2 in Examples 2 and 3) should be longer when the target pictures are visually degraded (because resumption should be comparatively difficult) than when they are not (because resumption should be comparatively easy). Experiment 2 presented the same conditions, but participants were now told not to resume with the name of the target picture. In this way, we controlled for possible effects of visual degradation on the perception detection of the change.