The Interplay Between Gesture and Speech in the Production of Referring Expressions: Investigating the Tradeoff Hypothesis
Topics in Cognitive Science 4 (2):232-248 (2012)
The tradeoff hypothesis in the speech–gesture relationship claims that (a) when gesturing gets harder, speakers will rely relatively more on speech, and (b) when speaking gets harder, speakers will rely relatively more on gestures. We tested the second part of this hypothesis in an experimental collaborative referring paradigm where pairs of participants (directors and matchers) identified targets to each other from an array visible to both of them. We manipulated two factors known to affect the difficulty of speaking to assess their effects on the gesture rate per 100 words. The first factor, codability, is the ease with which targets can be described. The second factor, repetition, is whether the targets are old or new (having been already described once or twice). We also manipulated a third factor, mutual visibility, because it is known to affect the rate and type of gesture produced. None of the manipulations systematically affected the gesture rate. Our data are thus mostly inconsistent with the tradeoff hypothesis. However, the gesture rate was sensitive to concurrent features of referring expressions, suggesting that gesture parallels aspects of speech. We argue that the redundancy between speech and gesture is communicatively motivated
|Keywords||Referring expressions Gesture–speech redundancy Speech production Gesture–speech tradeoff Gesture Iconic gestures Pointing|
|Categories||categorize this paper)|
References found in this work BETA
The Information Capacity of the Human Motor System in Controlling the Amplitude of Movement.Paul M. Fitts - 1954 - Journal of Experimental Psychology 47 (6):381.
Referring as a Collaborative Process.Herbert H. Clark & Deanna Wilkes-Gibbs - 1986 - Cognition 22 (1):1-39.
A Spreading-Activation Theory of Lemma Retrieval in Speaking.Ardi Roelofs - 1992 - Cognition 42 (1-3):107-142.
Using the Hands to Identify Who Does What to Whom: Gesture and Speech Go Hand‐in‐Hand.Wing Chee So, Sotaro Kita & Susan Goldin‐Meadow - 2009 - Cognitive Science 33 (1):115-125.
Word Effects in Speech Production: Retrieval of Syntactic Information and of Morphological Form.J. Jescheniak & W. Levelt - 1994 - Journal of Experimental Psychology 20:1-20.
Citations of this work BETA
Toward a Computational Psycholinguistics of Reference Production.Kees van Deemter, Albert Gatt, Roger P. G. van Gompel & Emiel Krahmer - 2012 - Topics in Cognitive Science 4 (2):166-183.
Imposing Cognitive Constraints on Reference Production: The Interplay Between Speech and Gesture During Grounding.Ingrid Masson‐Carro, Martijn Goudbeek & Emiel Krahmer - 2016 - Topics in Cognitive Science 8 (4):819-836.
Similar books and articles
Does the Hand Reflect Implicit Knowledge? Yes and No.Susan Goldin-Meadow & Martha Wagner Alibali - 1999 - Behavioral and Brain Sciences 22 (5):766-767.
Effects of Ambiguous Gestures and Language on the Time Course of Reference Resolution.Max M. Louwerse & Adrian Bangerter - 2010 - Cognitive Science 34 (8):1517-1529.
Gesture-First, but No Gestures?David McNeill, Bennett Bertenthal, Jonathan Cole & Shaun Gallagher - 2005 - Behavioral and Brain Sciences 28 (2):138-139.
Speech and Gesture Are Mediated by Independent Systems.Anna M. Barrett, Anne L. Foundas & Kenneth M. Heilman - 2005 - Behavioral and Brain Sciences 28 (2):125-126.
Speech and Gesture in Spatial Language and Cognition Among the Yucatec Mayas.Olivier le Guen - 2011 - Cognitive Science 35 (5):905-938.
From Past to Present: Speech, Gesture, and Brain in Present-Day Human Communication.Spencer D. Kelly - 2003 - Behavioral and Brain Sciences 26 (2):230-231.
The Mirror System Hypothesis Stands but the Framework is Much Enriched.Michael A. Arbib - 2005 - Behavioral and Brain Sciences 28 (2):149-159.
From Mouth to Hand: Gesture, Speech, and the Evolution of Right-Handedness.Michael C. Corballis - 2003 - Behavioral and Brain Sciences 26 (2):199-208.
A Zetetic's Perspective on Gesture, Speech, and the Evolution of Right-Handedness.Amir Raz & Opher Donchin - 2003 - Behavioral and Brain Sciences 26 (2):237-238.
Pumping for Gestural Origins: The Well May Be Rather Dry.Rick Dale, Daniel C. Richardson & Michael J. Owren - 2003 - Behavioral and Brain Sciences 26 (2):218-219.
Added to index2012-03-03
Total downloads85 ( #60,880 of 2,164,001 )
Recent downloads (6 months)1 ( #348,017 of 2,164,001 )
How can I increase my downloads?