This study explores the use of deictic gestures, vocalizations and words compared to content-loaded, or representational gestures and words in children’s early one- and two-element utterances. We analyze the spontaneous production of four children, observed longitudinally from 10–12 to 24–25 months of age, focusing on the components of children’s utterances, the information encoded, and the temporal relationship between gestures and vocalizations or words that were produced in combination. Results indicate that while the gestural and vocal modalities are meaningfully and (...) temporally integrated form the earliest stages, deictic and representational elements are unevenly distributed in the gestural vs. the vocal modality, and in one vs. two-element utterances. The findings suggest that while gestural deixis plays a primary role in allowing children to define and articulate their vocal productions, representational skills appear to be markedly more constrained in the gestural as compared to the vocal modality. (shrink)

People frequently gesture when problem-solving, particularly on tasks that require spatial transformation. Gesture often facilitates task performance by interacting with internal mental representations, but how this process works is not well understood. We investigated this question by exploring the case of mental abacus, a technique in which users not only imagine moving beads on an abacus to compute sums, but also produce movements in gestures that accompany the calculations. Because the content of MA is transparent and readily manipulated, (...) the task offers a unique window onto how gestures interface with mental representations. We find that the size and number of MA gestures reflect the length and difficulty of math problems. Also, by selectively interfering with aspects of gesture, we find that participants perform significantly worse on MA under motor interference, but that perceptual feedback is not critical for success on the task. We conclude that premotor processes involved in the planning of gestures are critical to mental representation in MA. (shrink)

Adults and children spontaneously produce gestures while they speak, and such gestures appear to support and expand on the information communicated by the verbal channel. Little research, however, has been carried out to examine the role played by gesture in the listener's representation of accumulating information. Do listeners attend to the gestures that accompany narrative speech? In what kinds of relationships between gesture and speech do listeners attend to the gestural channel? If listeners do attend to information received (...) in gesture, how is this information represented— is it 'tagged' as originating in the gestural channel? In this article research is described that addresses these questions. Results show that listeners do attend to information conveyed in gesture, when that information supplements or even contradicts the information conveyed by speech. And information received via gesture is available for retelling in speech. These results are taken to demonstrate that gesture is not taken by the listener to be epiphenomenal to the act of speaking, or a simple manual translation of speech. But they also suggest that the information conveyed in a discourse may be represented in a manner that is neither gesture nor language, although accessible to both channels. (shrink)

People frequently gesture when a word is on the tip of their tongue (TOT), yet research is mixed as to whether and why gesture aids lexical retrieval. We tested three accounts: the lexical retrieval hypothesis, which predicts that semantically related gestures facilitate successful lexical retrieval; the cognitive load account, which predicts that matching gestures facilitate lexical retrieval only when retrieval is hard, as in the case of a TOT; and the motor movement account, which predicts that any motor (...) movements should support lexical retrieval. In Experiment 1 (a between‐subjects study; N = 90), gesture inhibition, but not neck inhibition, affected TOT resolution but not overall lexical retrieval; participants in the gesture‐inhibited condition resolved fewer TOTs than participants who were allowed to gesture. When participants could gesture, they produced more representational gestures during resolved than unresolved TOTs, a pattern not observed for meaningless motor movements (e.g., beats). However, the effect of gesture inhibition on TOT resolution was not uniform; some participants resolved many TOTs, while others struggled. In Experiment 2 (a within‐subjects study; N = 34), the effect of gesture inhibition was traced to individual differences in verbal, not spatial short‐term memory (STM) span; those with weaker verbal STM resolved fewer TOTs when unable to gesture. This relationship between verbal STM and TOT resolution was not observed when participants were allowed to gesture. Taken together, these results fit the cognitive load account; when lexical retrieval is hard, gesture effectively reduces the cognitive load of TOT resolution for those who find the task especially taxing. (shrink)

Speakers frequently perform representational gestures to depict concepts in an iconic fashion. For example, a speaker may hold her index finger and thumb apart to indicate the size of a matchstick. However, the process by which a physical handshape is mentally transformed into abstract spatial information is not well understood. We present a series of experiments that investigate how people decode the physical form of an articulator to derive imaginary geometrical constructs, which we call “gesture form.” We provide (...) quantitative evidence for several key properties that play a role in this process. First, “profiling,” the ability to focus on a structural subunit within the complex form of the physical hand. Second, “perspective,” for which we show that one and the same handshape seen from different perspectives can lead to different spatial interpretations. Third, “selectivity,” the fact that gestures focus on specific spatial features at the expense of others. Our results provide a first step toward mapping out the process of how representational gestures make the communication of spatial information possible. (shrink)

Using a cross‐modal semantic priming paradigm, both experiments of the present study investigated the link between the mental representations of iconic gestures and words. Two groups of the participants performed a primed lexical decision task where they had to discriminate between visually presented words and nonwords (e.g., flirp). Word targets (e.g., bird) were preceded by video clips depicting either semantically related (e.g., pair of hands flapping) or semantically unrelated (e.g., drawing a square with both hands) gestures. The duration of gestures (...) was on average 3,500 ms in Experiment 1 but only 1,000 ms in Experiment 2. Significant priming effects were observed in both experiments, with faster response latencies for related gesture–word pairs than unrelated pairs. These results are consistent with the idea of interactions between the gestural and lexical representational systems, such that mere exposure to iconic gestures facilitates the recognition of semantically related words. (shrink)

We call those gestures “instrumental” that can enhance certain thinking processes of an agent by offering him representational models of his actions in a virtual space of imaginary performative possibilities. We argue that pointing is an instrumental gesture in that it represents geometrical information on one’s own gaze direction, and provides a ritualized template for initiating gaze coordination and joint attention. We counter two possible objections, asserting respectively that the representational content of pointing is not constitutive, but (...) derived from language, and that pointing directly solicits gaze coordination, without representing it. We consider two studies suggesting that attention and spatial perception are actively modified by one’s own pointing activity: the first study shows that pointing gestures help children link sets of objects to their corresponding number words; the second, that adults are faster and more accurate in counting when they point. (shrink)

Using a cross-modal semantic priming paradigm, both experiments of the present study investigated the link between the mental representations of iconic gestures and words. Two groups of the participants performed a primed lexical decision task where they had to discriminate between visually presented words and nonwords (e.g., flirp). Word targets (e.g., bird) were preceded by video clips depicting either semantically related (e.g., pair of hands flapping) or semantically unrelated (e.g., drawing a square with both hands) gestures. The duration of gestures (...) was on average 3,500 ms in Experiment 1 but only 1,000 ms in Experiment 2. Significant priming effects were observed in both experiments, with faster response latencies for related gesture–word pairs than unrelated pairs. These results are consistent with the idea of interactions between the gestural and lexical representational systems, such that mere exposure to iconic gestures facilitates the recognition of semantically related words. (shrink)

Research has shown that gesture production supports learning across a number of tasks. It is unclear, however, whether gesture production during encoding can support narrative recall, who gesture production benefits most, and whether certain types of gestures are more beneficial than others. This study, therefore, investigated the effect of gesture production during the encoding of a narrative on subsequent narrative recall, and whether individuals’ levels of verbal and nonverbal memory moderated this effect. Additionally, this study investigated (...) whether producing certain types of gestures during encoding was more beneficial than others. Participants (N = 90, Mage = 20.43) read aloud a narrative while under instruction to produce gestures, under no specific instruction to produce gestures, or were required to keep their hands behind their back to prevent them from gesturing. Participants completed measures assessing verbal and nonverbal memory. While gesture production during encoding benefitted narrative recall (as measured through specific questions), verbal memory moderated the effect, such that gesture production was more beneficial for individuals with higher than lower verbal memory. Furthermore, producing representational gestures during encoding benefitted recall of points in the narrative at which those gestures were produced, while beat gestures had no effect. Findings have implications for understanding the mechanisms underlying the links between gesture and learning, as well as practical implications in instructional settings. (shrink)

This study examined the performances of three groups of participants in interpreting metaphors in three different conditions: congruent gesture-prime conditions, opposite gesture-prime conditions, and no-prime conditions. In congruent gesture-prime conditions, each metaphor was preceded by the congruent gestural representation of metaphor schema. In opposite gesture-prime conditions, each metaphor was preceded by the opposite gestural representation of metaphor schema. The results showed that participants of congruent gesture-prime conditions had the best performance in interpreting metaphors, while participants (...) of opposite gesture-prime conditions had the worst performance. It is suggested that metaphor schema is an important part of metaphorical meaning of a metaphor. Therefore, when this part of meaning is activated by a gestural prime, metaphor comprehension is facilitated. Furthermore, when a schema is used to metaphorically describe a concept or an event, that schema and the neural network that represents it become one part of a larger interconnected neural network that represents the meaning of the metaphor. It could mean that the activation of one part of it could facilitate the activation of the whole of the network. Finally, it is emphasized that a gesture that depicts the schema of a metaphor is in fact the embodied realization of that metaphor. (shrink)

Most language learners have difficulties acquiring the phonemes of a second language (L2). Unfortunately, they are often judged on their L2 pronunciation, and segmental inaccuracies contribute to miscommunication. Therefore, we aim to determine how to facilitate phoneme acquisition. Given the close relationship between speech and co-speech gesture, previous work unsurprisingly reports that gestures can benefit language acquisition, e.g., in (L2) word learning. However, gesture studies on L2 phoneme acquisition present contradictory results, implying that both specific properties of gestures (...) and phonemes used in training, and their combination, may be relevant. We investigated the effect of phoneme and gesture complexity on L2 phoneme acquisition. In a production study, Dutch natives received instruction on the pronunciation of two Spanish phonemes, /u/ and /θ/. Both are typically difficult to produce for Dutch natives because their orthographic representation differs between both languages. Moreover, /θ/ is considered more complex than /u/, since the Dutch phoneme inventory contains /u/ but not /θ/. The instruction participants received contained Spanish examples presented either via audio-only, audio-visually without gesture, audio-visually with a simple, pointing gesture, or audio-visually with a more complex, iconic gesture representing the relevant speech articulator(s). Preceding and following training, participants read aloud Spanish sentences containing the target phonemes. In a perception study, Spanish natives rated the target words from the production study on accentedness and comprehensibility. Our results show that combining gesture and speech in L2 phoneme training can lead to significant improvement in L2 phoneme production, but both gesture and phoneme complexity affect successful learning: Significant learning only occurred for the less complex phoneme /u/ after seeing the more complex iconic gesture, whereas for the more complex phoneme /θ/, seeing the more complex gesture actually hindered acquisition. The perception results confirm the production findings and show that items containing /θ/ produced after receiving training with a less complex pointing gesture are considered less foreign-accented and more easily comprehensible as compared to the same items after audio-only training. This shows that gesture can facilitate task performance in L2 phonology acquisition, yet complexity affects whether certain gestures work better for certain phonemes than others. (shrink)

Our earliest tools are our bodies. Our hands raise and turn and toss and carry and push and pull, our legs walk and climb and kick allowing us to move and act in the world and to create the multitude of artifacts that improve our lives. The list of actions made by our hands and feet and other parts of our bodies is long. What is more remarkable is we turn those actions in the world into actions on thought through (...) gestures, language, and graphics, thereby creating cognitive tools that expand the mind. The focus here is gesture; gestures transform actions on perceptible objects to actions on imagined thoughts, carrying meaning with them rapidly, precisely, and directly. We review evidence showing that gestures enhance our own thinking and change the thought of others. We illustrate the power of gestures in studies showing that gestures uniquely change conceptions of time, from sequential to simultaneous, from sequential to cyclical, and from a perspective embedded in a timeline to an external perspective looking on a timeline, and by so doing obviate the ambiguities of an embedded perspective. We draw parallels between representations in gesture and in graphics; both use marks or actions arrayed in space to communicate more immediately than symbolic language. (shrink)

Pantomime has long been considered distinct from co-speech gesture. It has therefore been argued that pantomime cannot be part of gesture-speech integration. We examine pantomime as distinct from silent gesture, focusing on non-co-speech gestures that occur in the midst of children’s spoken narratives. We propose that gestures with features of pantomime are an infrequent but meaningful component of a multimodal communicative strategy. We examined spontaneous non-co-speech representational gesture production in the narratives of 30 monolingual English-speaking (...) children between the ages of 8- and 11-years. We compared the use of co-speech and non-co-speech gestures in both autobiographical and fictional narratives and examined viewpoint and the use of non-manual articulators, as well as the length of responses and narrative quality. The use of non-co-speech gestures was associated with longer narratives of equal or higher quality than those using only co-speech gestures. Non-co-speech gestures were most likely to adopt character-viewpoint and use non-manual articulators. The present study supports a deeper understanding of the term pantomime and its multimodal use by children in the integration of speech and gesture. (shrink)

Gestures can serve as external representations of abstract concepts which may be otherwise difficult to illustrate. Gestures often accompany verbal statement as an embodiment of mental models that augment the communication of ideas, concepts or envisioned shapes of products. A gesture is also an indicator of the subject and context of the issue under discussion. We argue that if gestures can be identified and formalized they can be used as a knowledge indexing and retrieval tool and can prove to (...) be useful access point into unstructured digital video data. We present a methodology and a prototype, called I-Gesture that allows users to (1) define a vocabulary of gestures for a specific domain, (2) build a digital library of the gesture vocabulary, and (3) mark up entire video streams based on the predefined vocabulary for future search and retrieval of digital content from the archive. I-Gesture methodology and prototype are illustrated through scenarios where it can be utilized. The paper concludes with results of evaluation experiments with I-Gesture using a test bed of design-construction projects. (shrink)

Developed through a series of conceptual analyses (Edmund Husserl, Vilém Flusser, Ludwig Wittgenstein and Walter Benjamin) and case studies (Fernando Lopes’s Belarmino and Jeff Wall’s Mimic), this article delves into the relationship between gesture, attunement and atmosphere and how it unfolds in photographic works dealing with urban space. The first section focuses on the role played by photography in the film Belarmino, which raises questions about both the representation of urban phenomena and issues related to expression and gesture (...) in boxing. The second section discusses Husserl’s thinking on image consciousness and his surprising reference to a “photograph” of a boxer, which reveals the relevance of his phenomenological approach when it comes to defining the aesthetic properties of gesture-images. The third section examines the principles of Flusser’s philosophy of gestures, focusing on the semantic field of attunement and its connection with various elements related to photography, gesture, moods and affects. The question of gesture in photography—both the gestures of the photographed and those of the photographer—can be articulated with the notions of attunement and atmosphere, which go beyond semiological, psychological and communicational approaches and are important for our understanding of aesthetic and artistic experiences. Finally, if photography is a privileged way of studying atmospheres and gestures (as suggested by the Benjaminian notion of optical unconscious) and their connection with the inner life of the subject, in relation to urban space this study often acquires an intersubjective, social and political dimension, as in Jeff Wall’s Mimic. (shrink)

The idea of ‘gesture’ is present in the philosophical world in various forms. All of them might find an important theoretical grounding in pragmatist philosophy, if we combine pragmatism with some French philosophies of mathematics and read it as a way out of the Kantian philosophy of representation. The paper uses the insights of Jean Cavaillès to set out the problem of the weakness of the epistemic Kantian defense of mathematical and logical thought. Cavaillès rejected the possible amendments to (...) Kant’s explanation provided by both Husserl and Bolzano and their heirs. He used the word ‘gesture’ in order to explain the activity of mathematicians who have to act synthetically, following rules, with some physical representation, and being aware of the possibility of failure. Cavaillès conceived the use of gesture as an alternative to the Heidegerian idea of event defended by Albert Lautman. The paper then follows the idea of gesture in the French philosophy of mathematics of Gilles Châtelet and Giuseppe Longo. Finally, the paper illustrates how Peirce’s study of Existential Graphs and the main insights of pragmatism complete Cavaillès’s idea by giving to gestures a phenomenological and semiotic structure. The pragmatist philosophy of gesture is thus a new way of overthrowing Kant’s philosophy of representation without surrendering to irrationalism. (shrink)

The present study investigates the types of verb and symbolic representational strategies used by 10 deaf signing adults and 13 deaf signing children who described in Italian Sign Language 45 video clips representing nine action types generally communicated by five general verbs in spoken Italian. General verbs, in which the same sign was produced to refer to several different physical action types, were rarely used by either group of participants. Both signing children and adults usually produced specific depicting predicates (...) by incorporating, through a representational strategy, the object and/or the modality of the action into the sign. As for the different types of representational strategies, the adults used the hand-as-object strategy more frequently than the children, who, in turn, preferred to use the hand-as-hand strategy, suggesting that different degrees of cognitive complexity are involved in these two symbolic strategies. Addressing the symbolic iconic strategies underlying sign formation could provide new insight into the perceptual and cognitive processes of linguistic meaning construction. The findings reported here support two main assumptions of cognitive linguistics applied to sign languages: there is a strong continuity between gestures and language; lexical units and depicting constructions derive from the same iconic core mechanism of sign creation. (shrink)

Human languages typically employ a variety of spatial metaphors for time (e.g., “I'm looking forward to the weekend”). The metaphorical grounding of time in space is also evident in gesture. The gestures that are performed when talking about time bolster the view that people sometimes think about regions of time as if they were locations in space. However, almost nothing is known about the development of metaphorical gestures for time, despite keen interest in the origins of space–time metaphors. In (...) this study, we examined the gestures that English‐speaking 6‐to‐7‐year‐olds, 9‐to‐11‐year‐olds, 13‐to‐15‐year‐olds, and adults produced when talking about time. Participants were asked to explain the difference between pairs of temporal adverbs (e.g., “tomorrow” versus “yesterday”) and to use their hands while doing so. There was a gradual increase across age groups in the propensity to produce spatial metaphorical gestures when talking about time. However, even a substantial majority of 6‐to‐7‐year‐old children produced a spatial gesture on at least one occasion. Overall, participants produced fewer gestures in the sagittal (front‐back) axis than in the lateral (left‐right) axis, and this was particularly true for the youngest children and adolescents. Gestures that were incongruent with the prevailing norms of space–time mappings among English speakers (leftward and backward for past; rightward and forward for future) gradually decreased with increasing age. This was true for both the lateral and sagittal axis. This study highlights the importance of metaphoricity in children's understanding of time. It also suggests that, by 6 to 7 years of age, culturally determined representations of time have a strong influence on children's spatial metaphorical gestures. (shrink)

Reasoning about bedrock abstract concepts such as time, number, and valence relies on spatial metaphor and often on multiple spatial metaphors for a single concept. Previous research has documented, for instance, both future-in-front and future-to-right metaphors for time in English speakers. It is often assumed that these metaphors, which appear to have distinct experiential bases, remain distinct in online temporal reasoning. In two studies we demonstrate that, contra this assumption, people systematically combine these metaphors. Evidence for this combination was found (...) in both directly elicited and spontaneous co-speech gestures about time. These results provide first support for the hypothesis that the metaphorical representation of time, and perhaps other abstract domains as well, involves the continuous co-activation of multiple metaphors rather than the selection of only one. (shrink)

Past research has sought to elucidate how speakers and addressees establish common ground in conversation, yet few studies have focused on how visual cues such as co-speech gestures contribute to this process. Likewise, the effect of cognitive constraints on multimodal grounding remains to be established. This study addresses the relationship between the verbal and gestural modalities during grounding in referential communication. We report data from a collaborative task where repeated references were elicited, and a time constraint was imposed to increase (...) cognitive load. Our results reveal no differential effects of repetition or cognitive load on the semantic-based gesture rate, suggesting that representational gestures and speech are closely coordinated during grounding. However, gestures and speech differed in their execution, especially under time pressure. We argue that speech and gesture are two complementary streams that might be planned in conjunction but that unfold independently in later stages of language production, with speakers emphasizing the form of their gestures, but not of their words, to better meet the goals of the collaborative task. (shrink)

When young children learn to use language, they start to use their hands in co‐verbal gesturing. There are, however, considerable differences between children, and it is not completely understood what these individual differences are due to. We studied how children at 4 years of age employ speech and iconic gestures to convey meaning in different kinds of spatial event descriptions, and how this relates to their cognitive abilities. Focusing on spontaneous illustrations of actions, we applied a semantic feature (SF) analysis (...) to characterize combinations of speech and gesture meaning and related them to the child's visual‐spatial abilities or abstract/concrete reasoning abilities (measured using the standardized SON‐R test). Results show that children with higher cognitive abilities convey significantly more meaning via gesture and less solely via speech. These findings suggest that young children's use of cospeech representational gesturing is positively related to their mental representation and reasoning abilities. (shrink)

This paper introduces a manual movement performed recurrently by German children in the age range of four to six. Based on the movement gestalt and its meaning, we termed it the Slapping movement. All forms identified in the data were performed with a communicative function, yet they showed different degrees of “gesturality.” To be more precise, we observed versions that clearly count as actions or gestures, but we also observed transitional forms between them. Based on a thorough analyses of form, (...) meaning, and context we determined variations of the Slapping gesture that showed different degrees of abstraction from action to gesture in a semiotic sense. These degrees are distinguished by modifications in the execution of the movement and different levels of form stability, environmental coupling, and representational complexity. (shrink)

This article proposes an analysis of the gestural production of people suffering from fluent or non-fluent aphasia in comparison with the gestures performed by speakers not suffering from any language disorder. The interest of such a study is twofold: firstly, it allows a better understanding of what happens to gestures during the numerous speech dysfluencies caused by the speech impairment. Among other things, it sheds light on the links between speech and gestures in multimodal face-to-face interaction. Despite the profusion of (...) work on aphasia, very little is still known about the characteristics of the gestures performed spontaneously by aphasic people. The study presented here is based on video examples from the AphasiaBank corpus to illustrate the specific features of the speech and gestures of atypical speakers and gives a more detailed description of the gestural production of people with aphasia. This knowledge can inform post-stroke rehabilitation procedures that rely on some form of multimodal speech therapy in order to adapt to the various communication needs of the patients. (shrink)

Language and speech depend on a relatively well defined neural circuitry, located predominantly in the left hemisphere. In this article, I discuss the origin of the speech circuit in early humans, as an expansion of an auditory-vocal articulatory network that took place after the last common ancestor with the chimpanzee. I will attempt to converge this perspective with aspects of the Mirror System Hypothesis, particularly those related to the emergence of a meaningful grammar in human communication. Basically, the strengthening of (...) auditory-vocal connectivity via the arcuate fasciculus and related tracts generated an expansion of working memory capacity for vocalizations, that was key for learning complex utterances. This process was concomitant with the development of a robust interface with visual working memory, both in the dorsal and ventral streams of auditory and visual processing. This enabled the bidirectional translation of sequential codes into hierarchical visual representations, through the development of a multimodal interface between both systems. (shrink)

Although children learn more when teachers gesture, it is not clear how gesture supports learning. Here, we sought to investigate the nature of the memory processes that underlie the observed benefits of gesture on lasting learning. We hypothesized that instruction with gesture might create memory representations that are particularly resistant to interference. We investigated this possibility in a classroom study with 402 second‐ and third‐grade children. Participants received classroom‐level instruction in mathematical equivalence using videos with or (...) without accompanying gesture. After instruction, children solved problems that were either visually similar to the problems that were taught, and consistent with an operational interpretation of the equal sign (interference), or visually distinct from equivalence problems and without an equal sign (control) in order to assess the role of gesture in resisting interference after learning. Gesture facilitated learning, but the effects of gesture and interference varied depending on type of problem being solved and the strategies that children used to solve problems prior to instruction. Some children benefitted from gesture, while others did not. These findings have implications for understanding the mechanisms underlying the beneficial effect of gesture on mathematical learning, revealing that gesture does not work via a general mechanism like enhancing attention or engagement that would apply to children with all forms of prior knowledge. (shrink)

This article explores the correlations between linguistic figurative features and their corresponding conceptual representations, by considering their respective continuities and discontinuities in language shift. I compare the figurative encoding of emotions in Kriol, a creole of northern Australia, with those of Dalabon, one of the languages replaced by this creole, with a particular focus on evidence from metaphorical gestures. The conclusions are three-fold. Firstly, the prominent figurative association between the body and the emotions observed in Dalabon is, overall, not matched (...) in Kriol. Secondly, although this association is not prominent in Kriol, it is not entirely absent. It surfaces where speakers are less constrained by linguistic conventions: in non-conventionalized tropes, and gestures in particular. Indeed, some of the verbal emotion metaphors that have disappeared with language shift are preserved as gestural metaphors. Thus, Kriol speakers endorse the conceptual association between emotions and the body, in spite of the lower linguistic incidence of this association. The third conclusion is that therefore, in language shift, conceptual figurative representations and linguistic figurative representations are independent of each other. The former can persist when the latter largely disappear. Conversely, the fact that speakers endorse a certain type of conceptual representation does not entail that they will use corresponding linguistic forms in the new language. The transfer of linguistic figurative representations seems to depend, instead, upon purely linguistic parameters. (shrink)

A Certain Gesture: Evnine's Batman Meme Project and Its Parerga! is an entirely original kind of work. It takes the form of commentaries on memes made with the image of Batman slapping Robin. The commentaries are written as if they were not authored by the same person who made the memes, allowing the author to consider himself and his work from the outside. The book defies genre by mixing discussions of philosophy, psychoanalysis, Judaism, language, and representation with self-writing and (...) autotheory. These are juxtaposed like the items in a cabinet of curiosities or topics in an analytic patient’s free association. Both pre-modern and post-modern in its inspiration, it is cerebral, playful, social, and intensely personal. It contains philosophy, including original philosophical research, but also explores new ways of doing and thinking about philosophy. (shrink)

Silent gesture is not considered to be linguistic, on par with spoken and sign languages. It is claimed that silent gestures, unlike language, represent events holistically, without compositional structure. However, recent research has demonstrated that gesturers use consistent strategies when representing objects and events, and that there are behavioral and clinically relevant limits on what form a gesture may take to effect a particular meaning. This systematicity challenges a holistic interpretation of silent gesture, which predicts that there (...) should be no stable form‐meaning correspondence across event representations. Here, we demonstrate to the contrary that untrained gesturers systematically manipulate the form of their gestures when representing events with and without a theme (e.g., Someone popped the balloon vs. Someone walked), that is, transitive and intransitive events. We elicited silent gestures and annotated them for manual features active in coding transitivity distinctions in sign languages. We trained linear support vector machines to make item‐by‐item transitivity predictions based on these features. Prediction accuracy was good across the entire dataset, thus demonstrating that systematicity in silent gesture can be explained with recourse to subunits. We argue that handshape features are constructs co‐opted from cognitive systems subserving manual action production and comprehension for communicative purposes, which may integrate into the linguistic system of emerging sign languages. We further suggest that nonsigners tend to map event participants to each hand, a strategy found across genetically and geographically distinct sign languages, suggesting the strategy's cognitive foundation. (shrink)

We call those gestures “instrumental” that can enhance certain thinking processes of an agent by offering him representational models of his actions in a virtual space of imaginary performative possibilities. We argue that pointing is an instrumental gesture in that it represents geometrical information on one’s own gaze direction, and provides a ritualized template for initiating gaze coordination and joint attention. We counter two possible objections, asserting respectively that the representational content of pointing is not constitutive, but (...) derived from language, and that pointing directly solicits gaze coordination, without representing it. We consider two studies suggesting that attention and spatial perception are actively modified by one’s own pointing activity: the first study shows that pointing gestures help children link sets of objects to their corresponding number words; the second, that adults are faster and more accurate in counting when they point. (shrink)

Foreign language education in the twenty-first century still teaches vocabulary mainly through reading and listening activities. This is due to the link between teaching practice and traditional philosophy of language, where language is considered to be an abstract phenomenon of the mind. However, a number of studies have shown that accompanying words or phrases of a foreign language with gestures leads to better memory results. In this paper, I review behavioral research on the positive effects of gestures on memory. Then (...) I move to the factors that have been addressed as contributing to the effect, and I embed the reviewed evidence in the theoretical framework of embodiment. Finally, I argue that gestures accompanying foreign language vocabulary learning create embodied representations of those words. I conclude by advocating the use of gestures in future language education as a learning tool that enhances the mind. (shrink)

Word learning is basic to foreign language acquisition, however time consuming and not always successful. Empirical studies have shown that traditional (visual) word learning can be enhanced by gestures. The gesture benefit has been attributed to depth of encoding. Gestures can lead to depth of encoding because they trigger semantic processing and sensorimotor enrichment of the novel word. However, the neural underpinning of depth of encoding is still unclear. Here, we combined an fMRI and a behavioral study to investigate (...) word encoding online. In the scanner, participants encoded 30 novel words of an artificial language created for experimental purposes and their translation into the subjects’ native language. Participants encoded the words three times visually, audiovisually, and by additionally observing semantically related gestures performed by an actress. Hemodynamic activity during word encoding revealed the recruitment of cortical areas involved in stimulus processing. In this study, depth of encoding can be spelt out in terms of sensorimotor brain networks that grow larger the more sensory modalities are linked to the novel word. Word retention outside the scanner documented a positive effect of gestures in a free recall test in the short-term. (shrink)

For humans, the ability to communicate and use language is instantiated not only in the vocal modality but also in the visual modality. The main examples of this are sign languages and gestures. Sign languages, the natural languages of Deaf communities, use systematic and conventionalized movements of the hands, face, and body for linguistic expression. Co-speech gestures, though non-linguistic, are produced in tight semantic and temporal integration with speech and constitute an integral part of language together with speech. The articles (...) in this issue explore and document how gestures and sign languages are similar or different and how communicative expression in the visual modality can change from being gestural to grammatical in nature through processes of conventionalization. As such, this issue contributes to our understanding of how the visual modality shapes language and the emergence of linguistic structure in newly developing systems. Studying the relationship between signs and gestures provides a new window onto the human ability to recruit multiple levels of representation in the service of using or creating conventionalized communicative systems. (shrink)

Vocabulary learning in a second language is enhanced if learners enrich the learning experience with self-performed iconic gestures. This learning strategy is called enactment. Here we explore how enacted words are functionally represented in the brain and which brain regions contribute to enhance retention. After an enactment training lasting 4 days, participants performed a word recognition task in the functional Magnetic Resonance Imaging (fMRI) scanner. Data analysis suggests the participation of different and partially intertwined networks that are engaged in higher (...) cognitive processes, i.e., enhanced attention and word recognition. Also, an experience-related network seems to map word representation. Besides core language regions, this latter network includes sensory and motor cortices, the basal ganglia, and the cerebellum. On the basis of its complexity and the involvement of the motor system, this sensorimotor network might explain superior retention for enactment. (shrink)

This study concerned the role of gestures that accompany discourse in deep learning processes. We assumed that co‐speech gestures favor the construction of a complete mental representation of the discourse content, and we tested the predictions that a discourse accompanied by gestures, as compared with a discourse not accompanied by gestures, should result in better recollection of conceptual information, a greater number of discourse‐based inferences drawn from the information explicitly stated in the discourse, and poorer recognition of verbatim of the (...) discourse. The results of three experiments confirmed these predictions. (shrink)

Deleuze’s film-philosophy makes much of the notion of virtual images in Bergson’s Matter and Memory, but in doing so he transforms a psycho-meta-physical thesis into a unBergsonian ontological one. In this essay, we will offer a corrective by exploring Bergson’s own explanation of the image as an “attitude of the body”—something that projects an actual, corporeal, and postural approach, not only to cinema, but also to philosophy. Indeed, just as Renoir famously said that “a director makes only one movie in (...) his life. Then he breaks it into pieces and makes it again,” so Bergson wrote that each philosopher only makes one “single point” throughout his or her whole career. And this one point, he then declares, is like a “vanishing image,” only one best understood as an attitude of the body. It is embodied image that underlies an alternative Bergsonian cinema of the actual and the body —one that we will examine through what Bergson’s has to say about “attitude” as well as “gesture” and “mime.” We will also look at it through a gestural concept enacted by a film, to be precise, the five remakes that comprise Lars von Trier’s and Jørgen Leth’s The Five Obstructions. This will bring us back to the idea of what it is that is being remade, both by directors and philosophers, in Renoir’s “one film” and Bergson’s singular “vanishing image” respectively. Is the “one” being remade an image understood as a representation, or is it a gesture, understood as a bodily movement? It is the latter stance that provides a wholly new and alternative view of Bergson’s philosophy of cinema. (shrink)

Acoustic speech output results from coordinated articulation of dozens of muscles, bones and cartilages of the vocal mechanism. While we commonly take the fluency and speed of our speech productions for granted, the neural mechanisms facilitating the requisite muscular control are not completely understood. Previous neuroimaging and electrophysiology studies of speech sensorimotor control has typically concentrated on speech sounds (i.e., phonemes, syllables and words) in isolation; sentence-length investigations have largely been used to inform coincident linguistic processing. In this study, we (...) examined the neural representations of segmental features (place and manner of articulation, and voicing status) in the context of fluent, continuous speech production. We used recordings from the cortical surface (electrocorticography (ECoG)) to simultaneously evaluate the spatial topography and temporal dynamics of the neural correlates of speech articulation that may mediate the generation of hypothesized gestural or articulatory scores. We found that the representation of place of articulation involved broad networks of brain regions during all phases of speech production: preparation, execution and monitoring. In contrast, manner of articulation and voicing status were dominated by auditory cortical responses after speech had been initiated. These results provide a new insight into the articulatory and auditory processes underlying speech production in terms of their motor requirements and acoustic correlates. (shrink)

We review major developmental evidence on the continuity from action to gesture to word and sign in human children, highlighting the important role of caregivers in the development of multimodal communication. In particular, the basic issues considered here and contributing to the current debate on the origins and development of the language-ready brain are: (1) links between early actions, gestures and words and similarities in representational strategies; (2) importance of multimodal communication and the interplay between gestures and spoken (...) words; (3) interconnections between early actions, gestures and signs. The innovation of this report is in connecting these themes together to relevant findings from studies on children between 6 and 36 months of age and highlighting interesting parallels in studies on ape communicative behavior. (shrink)

In this introduction to the special issue on time and viewpoint in narrative discourse, we highlight the central contributions of the issue concerning the relation between the linguistic construal and cognitive representation of time and viewpoint. We explain how linguistic and gestural cues guide the representation of narrative time progression and argue that this representation involves various cognitive operations regulating the alignment between the viewpoints of narrator, addressee, and narrative characters. These operations are steered by a variety of linguistic phenomena, (...) including verb tense, adverbs, demonstratives, reduplication and negation, as well as body partitioning. We discuss how genre characteristics influence the relation between the linguistic and cognitive representation of time and viewpoint. Finally, we explain how the analyses presented in the various papers both demonstrate and clarify how the cognitive representation of time in narrative discourse relies on the human capacity to simultaneously manage multiple viewpoints, leading to the conclusion that time and viewpoint are closely related in the representation of narrative discourse. (shrink)