Elsevier

Cognition

Volume 111, Issue 1, April 2009, Pages 55-71
Cognition

Discourse-mediation of the mapping between language and the visual world: Eye movements and mental representation

https://doi.org/10.1016/j.cognition.2008.12.005Get rights and content
Under a Creative Commons license
open access

Abstract

Two experiments explored the mapping between language and mental representations of visual scenes. In both experiments, participants viewed, for example, a scene depicting a woman, a wine glass and bottle on the floor, an empty table, and various other objects. In Experiment 1, participants concurrently heard either ‘The woman will put the glass on the table’ or ‘The woman is too lazy to put the glass on the table’. Subsequently, with the scene unchanged, participants heard that the woman ‘will pick up the bottle, and pour the wine carefully into the glass.’ Experiment 2 was identical except that the scene was removed before the onset of the spoken language. In both cases, eye movements after ‘pour’ (anticipating the glass) and at ‘glass’ reflected the language-determined position of the glass, as either on the floor, or moved onto the table, even though the concurrent (Experiment 1) or prior (Experiment 2) scene showed the glass in its unmoved position on the floor. Language-mediated eye movements thus reflect the real-time mapping of language onto dynamically updateable event-based representations of concurrently or previously seen objects (and their locations).

Keywords

Sentence comprehension
Eye movements
Visual scene interpretation
Situation models

Cited by (0)