Abstract
The ubiquitous human practice of spontaneously gesturing while speaking demonstrates the embodiment, embeddedness, and sociality of cognition. The present essay takes gestural practice to be a paradigmatic example of a more general claim: human cognition is social insofar as our embedded, intelligent, and interacting bodies select and construct meaning in a way that is intersubjectively constrained and defeasible. Spontaneous co-speech gesture is markedly interesting because it at once confirms embodied aspects of linguistic meaning-making that formalist and linguistic turn-type philosophical approaches fail to appreciate, and it also forefronts intersubjectivity as an inherent and inherently normative dimension of communicative action. Co-speech hand gestures, as linguistically meaningful speech acts, demonstrate both sedimentation and spontaneity (in the sense of Maurice Merleau-Ponty’s dialectic of linguistic expression (2002)), or features of convention and nonconvention in a Gricean sense (1989). Yet neither pragmatic nor classic phenomenological approaches to communication can accommodate the practice of co-speech hand gesturing without some rehabilitation and reorientation. Pragmatic criteria of intersubjectivity, normativity, and rationality need to confront the non-propositional and nonverbal meaning-making of embodied encounters. Phenomenological treatments of expression and intersubjectivity must consider the normative nature of high-order social practices like language use. Reciprocally critical exchanges between these traditions and gesture studies yield an improved philosophy that treats language as a multi-modal medium for collaborative meaning achievement. The proper paradigm for these discussions is found in enactive approaches to social cognition. Co-speech hand gestures are first and foremost emergent elements of social interaction, not the external whirring of an isolated internal consciousness. In contrast to current literature that frequently presents gestures as uncontrollable bodily upsurge or infallible imagistic phenomenon that drives and dances with verbal or “linguistic” convention (McNeill 1992, 2005), I suggest that we study gestures as dynamic, embodied, and shared tools for collaborative sense-making.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The author would like to thank Dr. Emma R. Jones, George Fourlas, and an anonymous reviewer for their great help and suggestions in revising this article.
For example, Strawson (2004), Gallese (2005), Stawarska (2006), De Jaegher and Di Paolo (2007), Fuchs and de Jaegher (2009), and Steiner and Stewart (2009) extend aspects of the 4E paradigm (Menary 2010, particularly enactive treatments such as Hutto (2005), Thompson (2005), and Simpson (2010)) to encompass sociality in explaining embodied cognition.
Throughout this work I argue yet more strongly that co-speech hand gestures are linguistic. This is done in deliberate effort to broaden our understanding of what languaging activities actually involve, as one anonymous reviewer helpfully summarizes my point. The goal here is not to lose important distinctions by broadening a term; rather, the goal is to correct a shocking oversight in the received way scholars carve up phenomena. My view is perhaps usefully located in relation to contemporary pragmatists such as Mark Johnson and Colin Koopman, who point out that there are meaningful human practices that are normatively structured—subject to criticism and defense, success and failure—which are non-linguistic (Johnson 2007, 34, 207–209; Koopman 2011, 75). Agreeing with Johnson and Koopman, my goal is to reserve a place for language use as a sub-region of human practices of meaning achievement and locate co-speech gesture as belonging to this place. Within the hierarchy of meaning-making practices and communicative acts that humans engage in, co-speech gestures occur with speech and achieve their meaning in close interaction with speech. I can communicate by throwing things, and this is probably not well-classified as a linguistic act. But surely pointing to a book is closer to the act of verbally bringing someone’s attention to that book than it is to throwing the book at them.
Thanks to anonymous reviewer for bringing my attention to several of these references.
See, in particular, Jürgen Streeck (2008). Complicating the traditional yet unsatisfactory understanding of iconicity as resting on perceived similarity, Jürgen Streeck identifies and analyzes a variety of practices by which gesturers achieve an interpretation in depiction (2008). Streeck names 12 methods by which hand gestures construe something as something for their receivers (2008, 292–295). These gestures make sense to participants immediately as the hand motions transparently give way to the selected schemata or features they enact. For Streeck, the “pictorial language” by which gestures construe consists “of schematized acts of making, handling, drawing, and so on: whatever is depicted—things, inanimate processes, actions—is depicted and at the same time analyzed in terms of manual acts. Knowledge of these acts… is not in the first place knowledge of the gesture methods (or gestures’ meanings), but of ways of acting in the material world” (2008, 298–299). Common practices and familiar action sequences in a shared world, rather than formal resemblance or mirroring, thus enable our understanding of depictive gestures.
Grice (1989, 28) gives the examples of fixing a car or baking as activities in which expectations for others’ rational and helpful contributions holds. Austin saw all speaking as an act. Austin writes, “There are a great many devices that can be used for making clear, even at the primitive level, what act it is we are performing when we say something—the tone of voice, cadence, gesture—and above all we can rely upon the nature of the circumstances, the context in which the utterance is issued” (1961, 231). For Austin, the conventions guiding how we perform utterances or “do things with words” are importantly relative to a society of language users. Austin explains “The social habits of the society may considerably affect the question of which performative verbs are evolved and which, sometimes for rather irrelevant reasons, are not” (1961, 232). Clearly Austin’s sense of “convention” is never simply or solely located in grammar or literal sentence meaning.
For example, consider Sperber and Wilson’s neo-Gricean Relevance Theory (1986), discussed below. Much like Searle’s famous elaboration of Austin, performance and deep social context is left out in favor of an individualized, interiorized, competence-based representational account of cognition.
Consider Adam Kendon’s point: In contrast to explanation of speech–gesture co-expressiveness according to a dialectic processing plan, Kendon suggests “that the conjunction of the stroke with the informational center of the spoken phrase is something that the speaker achieves. In creating an utterance that uses both modes of expression, the speaker creates an ensemble in which gesture and speech are employed together as partners in a single rhetorical enterprise” (2004, 127).
In his later “Retrospective Epilogue,” Grice notes, “What I have been calling conversational implicature is just those assumptions which have to be attributed to a speaker to justify him in regarding a given sequence of lower-order speech-acts as being rationalized by their relation to a conventionally indexed higher-order speech act” (1989, 370).
Compare a recent study by Ivan Enrici, Mauro Adenzato, Stefano Cappa, Bruno G. Bara, and Marco Tettamanti, which argues that “Human communicative competence is based on the ability to process a specific class of mental states, namely, communicative intention,” and recruits fMRI results to argue for a neural network that is used in processing communicative intention and that is modality-neutral yet accessed differently through different modalities (2011). This work seeks the neural correlates for intention processing in communication, which Sperber and Wilson presume but have not studied in a cognitive neuroscience paradigm. Yet the type of gestures they seek to add as a modality of communicative interaction are primarily emblematic, and the study measures reactions to pictures and sentences, which seems a bit thin for a “full communicative exchange” situation (2011, 2429). They conclude that communicative context transcends modality, but define communicative context by the presence of intentions to communicate, such that intentions as mental states remain a presupposed, unexplained explainer.
In a recent review of Wharton’s book, Kensy Cooperrider notes in criticism, “It seems that gesture—perhaps preeminently among the nonverbal behaviors Wharton discusses—challenges the ease of disentangling nature and convention, biology and culture” (Cooperrider 2011, 87).
It is important to note that this sort of normativity (rational interpretation) is not the only source of convention that conditions gesture meaning. We also recognize certain kinds of gestures as gestures, rather than other types of actions (Kendon 1980, 208). As Streeck writes in describing what are typically seen as “iconic” gestural representations, “Depiction is always a matter of convention… Whether I recognize a cluster of paint particles or a sequence of motions in the limbs as a likeness of an object or not is a matter of the methods by which these images have been made, and whether these methods are part of my cultural repertoire” (2009, 120). Thanks to an anonymous reviewer for pointing out the slippage in Enfield’s use of ‘sign,’ quoted above.
While not included in exegetical discussion here, the communicative action theories found in the work of Jürgen Habermas (1981), as well as Robert Brandom’s inferential pragmatist account of semantics (1994), require at least the potential for explicit propositionality in order for an act to be rational (and therefore communicative, on this view).
A similar conclusion is reached in Hanne De Jaegher and Ezequiel Di Paolo’s analysis of an example from Currie in which a woman nonverbally draws her partner’s attention and orients him generally to fully appreciate a moment she is experiencing. De Jaegher and Di Paolo write, “…it is through a process of coordination and modulation of sense-making activities that the orientee is directly affected by the orienter’s intentions and sense-making and therefore he does not need to figure out what these intentions are in order to respond accordingly. A coordinated response already embodies a practical understanding” (2007, 499).
As Dan Zahavi blends these perspectives: “The very possibility of intersubjectivity is rooted in the bodily constitution of subjectivity” (Zahavi 2005, 163).
“Over-emphasis on the latter skills [explanation and prediction] has led most of contemporary social cognitive science to paint a picture of individuals who have to work out each other’s minds much like they do with scientific problems. In this view, what counts as ‘social’ differs from non-social problem solving merely as a matter of degree” (De Jaegher and Di Paolo 2007, 486).
Even Jürgen Habermas, who offers quite a disembodied and formalist account of communicative action, specifies three kinds of world relation achieved in communication action. He offers a broad, “phenomenological” model, which includes three ways (objective, social, and subjective) in which we thematize an always already given, pre-thematic, shared lifeworld (1981, 83). On this phenomenological model, “…rational expressions have the character of meaningful actions, intelligible in their context, through which the actor relates to something in the objective world. The conditions of validity of symbolic expressions refer to a background knowledge intersubjectively shared by the communication community” (1981, 13).
Following Scheler’s critique of empathy and the problem of other minds, Zahavi writes, “we should avoid construing the mind as something visible to only one person and invisible to everyone else. The mind is not something exclusively inner, something cut off from the body and the surrounding world, as if psychological phenomena would remain precisely the same even without bodily and linguistic expressions” (2005, 152).
A wonderful, detailed example of how “the audience’s orientation to or away from the speaker determines the fate of the gestures that the speaker makes” is found in Streeck (1994, 257ff). Streeck describes a case in which an artist attempts to explain her exhibit to an audience of politicians visiting her art opening. At first, the politicians stare fixedly at the exhibit and miss her gestures, which grow increasingly small and half-hearted. When she notices the gaze of a member of an utterance is directed at her, the artist begins gesturing again, this time much more boldly and symbolically (259–265). Streeck’s description of this event gives us some fledgling criteria for gesture failure (“…they neither merge into one another nor combine into complex constructions. They remain isolated, bounded simple events” (259)) and success (the gestures “becoming structures in space that are set up in a sequence of preparation and stroke…” (262)).
See for example the conversation between Hanne De Jaegher and Shaun Gallagher on how best to frame what goes on in social interaction (De Jaegher 2009a, b). As Ezequiel Di Paolo suggests in a recent interview, “We’re still facing the messy beginnings in this [enactive] approach, still exploring our precursors, finding affinities with research lines in different disciplines that are enactive in all but name and clarifying open issues, getting at better foundations” (Interview with Ezequiel Di Paolo. New APPS: Art, Politics, Philosophy, Science. 1 Jun. 2011. Web.)
“Without ever denying the eminently intersubjective character of language, phenomenologists have often endeavored to unearth pre- or extra-linguistic forms of intersubjectivity, be it in simple perception or in tool-use, in emotions, drives, or body-awareness” (Zahavi 2005, 176). Zahavi takes this to be “decisively different” in approach from Habermas.
Notably, Beata Stawarska develops a dialogical phenomenology (2009a) that engages with empirical work on infant–caregiver interactions. See also Stawarska 2009b. While gesturing as Stawarska discusses is not paradigm co-speech gesturing, as pre-linguistic infants are involved, this work assists in defeating the theory-theory of mind and paving a way for the sort of inquiry being called for here.
In an often referenced example, a participant in McNeill’s lab re-tells the narrative of a Sylvester and Tweety cartoon wherein Tweety drops a bowling ball into the drainpipe of which Sylvester is concurrently climbing up the interior. The speaker’s sentence expresses that Tweety takes the bowling “ball and drops it down the drainpipe.” She makes a symmetrical two-handed gesture with palms loosely curved and facing down. The downward stroke of the gesture is synchronous with ‘down’. Importantly, the gesture stroke does not coincide with the verb ‘drops’, but is withheld to co-occur with ‘down’. The gesture is withheld because the core concept to be accomplished in this instance, according to McNeill, is what the bowling ball was doing and how it pushed Sylvester down a drainpipe. Though Tweety was still the agent in the utterance according to the verbal information, the gesture aided in transitioning to an understanding of the bowling ball as the real agentive force and ‘it down’ as the true “anchor” of the sentence (McNeill 2005, 122). In this way, the gesture manifests what the speaker takes as the important point requiring emphasis in that moment of the dialogue.
Furthermore, it is not clear that Merleau-Ponty would conscience a clean dialectical separation of the conventional (non-natural) and nonconventional (natural) within us. As he writes in The Structure of Behavior, “Man is not a rational animal. The appearances of reason and mind do not leave intact a sphere of self-enclosed instincts in man” (2006, 181). For Merleau-Ponty, the acting body is always discriminating and taking as, whether in speech, perception, or gesture.
“Action is the action of subjects; it is the action of minded individuals” (Zahavi 2005, 161). As Etienne Bimbenet describes the defining ambiguity of perception in Merleau-Ponty’s account, “It turns out, and this is the ultimate point, that there is in perception as much passivity as there is spontaneity, or that perception is a feeling at the same time that it is a thought” (Bimbenet 2009, 73).
A narrow sense of linguistic convention can yield vexing results in empirical gesture scholarship, for instance a recent study that investigates whether a hand gesture is conventional or motivated by a conceptual metaphor, as if these are mutually exclusive options (Parill 2008). An overly rigid, overly dichotomous view of imagistic and spontaneous thinking as being opposed to “linguistic” thinking wrongly presents some aspects of cognition as sheltered, personal, and pure of culturally shared habituation and sedimentation.
Recall Merleau-Ponty’s discussion of the angry Japanese person who smiles (2002, 219).
For example, gestures unwittingly give away our lies (Franklin 2007), while speech–gesture mismatches may “…point to a state of overload, in which the speaker’s emotional content exceeds the means of expression, and marks the search of this means of expression by the speaker” (Waisman 2010, 173). Such explanations logically fall out of a theory of speech–gesture interaction that ascribes to gesture all of the personal, idiosyncratic, and nonconventional aspects of cognition and communication. See McNeill and Duncan (2000) for a clear statement of the “window” view.
See Streeck’s more recent writings (2009, 2010) for a careful identification of six gestural ecologies, which on his account are kinds of world-relation accomplished through hand gestures made in conversation and coupled with local environments in different ways. Streeck writes, “…this heuristic enables us to take note of the fact that hand gestures not only embody meaning and mediate communication in heterogeneous ways, but also bring the communicating body in contact with the world in a variety of distinct modes” (2010, 226). While beyond the scope of the current essay, I find Streeck’s recent formulations promising for dialogue with someone like Habermas. Following this lead, we might say that gesture ecologies’ various ways of relating to the world in communicative action are so many enactments of rationality. Streeck’s work, both earlier and later, is particularly attuned to the recipient-designed nature of gestures, demonstrating that gestures require intersubjective recognition—one of Habermas’s criteria for rational attempts. Streeck’s example of the artist ceasing and then redesigning her gestures for her audience show that gesture attempts can fail (another Habermasian criterion).
References
Allen, B. (2008). Artifice and Design: Art and Technology in Human Experience. Ithaca: Cornell University Press.
Austin, J. L. (1961). Philosophical Papers. Oxford: Clarendon.
Bara, B. G., & Douthwaite, J. (2010). Cognitive Pragmatics: The Mental Processes of Communication. Cambridge, Massachusetts: The MIT Press.
Beata Stawarska 2009a. Between you and I: dialogical phenomenology. Ohio University Press.
Beata Stawarska & Meeting. 2006.: International Association for Phenomenology and the Cognitive Sciences; Intersubjectivity and embodiment Springer; 2006.
Bimbenet, E. (2009). Merleau-Ponty and the quarrel over the conceptual contents of perception. Trans. Robin Muller. Graduate Faculty Philosophy Journal, 30(1), 59–78.
Brandom, R. (1994). Making It Explicit: Reasoning, Representing and Discursive Commitment. Cambridge: Harvard University Press.
Calbris, G. (1990). The Semiotics of French Gestures. Bloomington: Indiana University Press.
Cienki, A., & Müller, C. (2008). Metaphor, gesture, and thought. In R. W. Gibbs Jr. (Ed.), The Cambridge Handbook of Metaphor and Thought. Cambridge: Cambridge University Press.
Cooperrider, K. (2011). Review of Pragmatics and Nonverbal Communication, by Tim Wharton (2009). Gesture, 11(1), 81–88.
De Jaegher, H. (2009a). Social understanding through direct perception? Yes, by interacting. Consciousness and Cognition, 18(2), 535–542.
De Jaegher, H. (2009b). What made me want the cheese? A reply to Shaun Gallagher and Dan Hutto. Consciousness and Cognition, 18(2), 549–550.
De Jaegher, H., & Di Paolo, Ezequiel. (2007). Participatory sense-making: An enactive approach to social cognition. Phenomenology and the Cognitive Sciences, 6, 485–507.
Egginton, W., & Sandbothe, M. (2004). The Pragmatic Turn in Philosophy: Contemporary Engagements Between Analytic and Continental Thought. Albany: State University of New York Press.
Enfield, N. J. (2009). The Anatomy of Meaning: Speech, Gesture, and Composite Utterances. Cambridge: Cambridge University Press.
Enrici, I., Adenzato, M., Cappa, S., Bara, B. G., & Tettamanti, M. (2011). Intention processing in communication: a common brain network for language and gestures. Journal of Cognitive Neuroscience, 23(9), 2415–2431.
Ezequiel Di Paolo. New APPS: Art, Politics, Philosophy, Science. 1 Jun. 2011. Web.
Franklin, A. (2007). Blending in deception: tracing output back to its source. In E. T. Levy, S. D. Duncan, & J. Cassell (Eds.), Gesture and the Dynamic Dimension of Language: Essays in Honor of David McNeill (pp. 99–109). Amsterdam: John Benjamins.
Fuchs, T., & de Jaegher, H. (2009). Enactive intersubjectivity: Participatory sense-making and mutual incorporation. Phenomenology and the Cognitive Sciences, 4(8), 465–486.
Gallese, V. (2005). Embodied simulation: From neurons to phenomenal experience. Phenomenology and the Cognitive Sciences, 1(4), 23–48.
Gendlin, E. (1962). Experiencing and the creation of meaning: A philosophical and psychological approach to the subjective. Evanston: Northwest University Press.
Goodwin, C. (2006). Human sociality as mutual orientation in a rich interactive environment. In N. J. Enfield & S. C. Levinson (Eds.), Roots of Human Sociality: Culture, Cognition, and Interaction. Oxford: Berg.
Goodwin, C., & Goodwin, M. H. (1986). Gesture and coparticipation in the activity of searching for a word. Semiotica, 62(1–2), 51–76.
Grice, H. P. (1989). Studies in the Way of Words. Cambridge: Harvard University Press.
Habermas, J. (1981). A Theory of Communicative Action (Vol. 1). Boston: Beacon.
Hutto, D. (2005). Knowing What? Radical Versus Conservative Enactivism. Phenomenology and the Cognitive Sciences, 4(4), 389–405.
Johnson, M. (2007). The Meaning of the Body: Aesthetics of Human Understanding. Chicago: University of Chicago Press.
Kendon, A. (1980). Gesticulation and speech: Two aspects of the process of utterance. In M. R. Key (Ed.), Nonverbal Communication and Language (pp. 207–227). The Hague: Mouton.
Kendon, A. (2004). Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press.
Kita, S. (2009). Cross-cultural variation of speech-accompanying gesture: A review. Language and Cognitive Processes, 24(2), 145–167.
Koopman, C. (2011). Rorty’s linguistic turn: Why (more than) language matters to philosophy. Contemporary Pragmatism, 8(1), 61–84.
Lakoff, G., & Johnson, M. (1999). Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought. New York: Basic Books.
McDowell, J. H. (1994). Mind and World. Cambridge: Harvard University Press.
McNeill, D. (1992). Hand and Mind: What Gestures Reveal About Thought. Chicago: The University of Chicago Press.
McNeill, D. (2005). Gesture & Thought. Chicago: University of Chicago Press.
McNeill, D., & Duncan, S. D. (2000). Growth points in thinking-for-speaking. In D. McNeill (Ed.), Language and Gesture. Cambridge: Cambridge University Press.
Menary, R. (2010). Introduction to the special issue on 4E cognition. Phenomenology and the Cognitive Sciences, 4(9), 459–463.
Merleau-Ponty, M. (2002). Phenomenology of Perception. London: Routledge Classics.
Merleau-Ponty, M. (2006). The structure of behavior. Trans. Alden L. Fisher. Pittsburgh: Duquesne University Press.
Müller, C. (2007). A dynamic view of metaphor, gesture and thought. In E. T. Levy, S. D. Duncan, & J. Cassell (Eds.), Gesture and the Dynamic Dimension of Language: Essays in Honor of David McNeill. Amsterdam: John Benjamins.
Parrill, F. (2008). Form, meaning, and convention: A comparison of metaphoric gesture with an emblem. In A. Cienki & C. Müller (Eds.), Metaphor and Gesture. Amsterdam: John Benjamins Publishing Company.
Rorty, R. (1979). Philosophy and the Mirror of Nature. Princeton: Princeton University Press.
Rorty, R. (1989). Contingency, Irony, Solidarity. Cambridge: Cambridge University Press.
Simpson, D. (2010). Language and know-how. Phenomenology and the Cognitive Sciences, 4(9), 629–643.
Sperber, D., & Wilson, D. (1986). Relevance: Communication and Cognition. Cambridge: Harvard University Press.
Sperber, D., & Wilson, D. (2002). Pragmatics, modularity and mind-reading. Mind and Language, 17(1), 3–23.
Stawarska, B. (2009). Merleau-Ponty and Sartre in response to cognitive studies of facial imitation. Philosophy Compass, 4(2), 312–328.
Steiner, P., & Stewart, J. (2009). From autonomy to heteronomy (and back): The enaction of social life. Phenomenology and the Cognitive Sciences, 4(8), 527–550.
Stewart, J. R., Gapenne, O., & Di Paolo, E. (2010). Enaction: Toward a New Paradigm for Cognitive Science. Cambridge: MIT.
Strawson, G. (2004). Real intentionality. Phenomenology and the Cognitive Sciences, 3(3), 287–313.
Streeck, J. (1993). Gesture as communication I: Its coordination with gaze and speech. Communication Monographs, 60(4), 275–299.
Streeck, J. (1994). Gesture as communication II: The audience as co-author. Research on Language and Social Interaction, 27(3), 239–267.
Streeck, J. (2008). Depicting by gesture. Gesture, 8(3), 285–301.
Streeck, J. (2009). Gesturecraft. Amsterdam: John Benjamins Publishing Co.
Streeck, J. (2010). New Adventures in Language and Interaction. Amsterdam: John Benjamins Publishing Co.
Sweetser, E., & Sizemore, M. (2008). Personal and interpersonal gesture spaces: Functional contrasts in language and gesture. In A. Tyler, Y. Kim, & M. Takada (Eds.), Language in the Context of Use: Discourse and Cognitive Approaches to Language. New York: Mouton de Gruyter.
Thompson, E. (2005). Sensorimotor subjectivity and the enactive approach to experience. Phenomenology and Cognitive Sciences, 4(4), 407–427.
Waisman, O. S. (2010). Body, Language and Meaning in Conflict Situations: A Semiotic Analysis of Gesture-Word Mismatches in Israeli-Jewish and Arab Discourse. Amsterdam/Philadelphia: John Benjamins Publishing Company.
Wharton, T. (2009). Pragmatics and Nonverbal Communication. Cambridge: Cambridge University Press.
Zahavi, D. (2005). Subjectivity and Selfhood: Investigating the First Person Perspective. Cambridge: MIT.
Zahavi, D. (2008). Simulation, perception, empathy. Consciousness and cognition, 17(2).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Cuffari, E. Gestural sense-making: hand gestures as intersubjective linguistic enactments. Phenom Cogn Sci 11, 599–622 (2012). https://doi.org/10.1007/s11097-011-9244-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11097-011-9244-9