We assessed how synchronous speech listening and lipreading affects speech recognition in acoustic noise. In simple audiovisual perceptual tasks, inverse effectiveness is often observed, which holds that the weaker the unimodal stimuli, or the poorer their signal-to-noise ratio, the stronger the audiovisual benefit. So far, however, inverse effectiveness has not been demonstrated for complex audiovisual speech stimuli. Here we assess whether this multisensory integration effect can also be observed for the recognizability of spoken words. To that end, we presented (...) audiovisual sentences to 18 native-Dutch normal-hearing participants, who had to identify the spoken words from a finite list. Speech-recognition performance was determined for auditory-only, visual-only (lipreading) and auditory-visual conditions. To modulate acoustic task difficulty, we systematically varied the auditory signal-to-noise ratio. In line with a commonly-observed multisensory enhancement on speech recognition, audiovisual words were more easily recognized than auditory-only words (recognition thresholds of -15 dB and -12 dB, respectively). We here show that the difficulty of recognizing a particular word, either acoustically or visually, determines the occurrence of inverse effectiveness in audiovisual word integration. Thus, words that are better heard or recognized through lipreading, benefit less from bimodal presentation. Audiovisual performance at the lowest acoustic signal-to-noise ratios (45%) fell below the visual recognition rates (60%), reflecting an actual deterioration of lipreading in the presence of excessive acoustic noise. This suggests that the brain may adopt a strategy in which attention has to be divided between listening and lipreading. (shrink)

This study adopted a change detection task to investigate whether and how recognition intent affects the construction of orthographic representation in visual word recognition. Chinese readers and nonreaders detected color changes in radical components of Chinese characters. Explicit recognition demand was imposed in Experiment 2 by an additional recognition task. When the recognition was implicit, a bias favoring the radical location informative of character identity was found in Chinese readers , but not nonreaders . (...) With explicit recognition demands, the effect of radical location interacted with radical function and word frequency . An estimate of identification performance under implicit recognition was derived in Experiment 3. These findings reflect the joint influence of recognition intent and orthographic regularity in shaping readers’ orthographic representation. The implication for the role of visual attention in word recognition was also discussed. (shrink)

Early vision provides general information about the environment that can be used for motor control or navigation and more specialized information that can be used for object recognition. The general information is likely to be insensitive to cognitive factors, but this may not be entirely true for the information used in model-based recognition.

Individuals with Down syndrome (DS) are widely believed to possess considerable socialization strengths. However, the findings on social cognition capabilities are controversial. In the present study, we investigated whether individuals with DS exhibit shortage in face tuning, one of the indispensable components of social cognition. For this purpose, we implemented a recently developed Face-n-Food paradigm with food-plate images composed of food ingredients such as fruits and vegetables. The key benefit of such ‘face like non-face’ images is that single elements do (...) not facilitate face processing. In a spontaneous recognition task, 25 children with DS aged 9 to 18 years were presented with a set of Face-n-Food images bordering on the Giuseppe Arcimboldo style. The set of images was administered in a predetermined order from the least to most resembling a face. In DS individuals, thresholds for recognition of the Face-n-Food images as a face were drastically higher as compared not only with typically developing controls, but also with individuals with autistic spectrum disorders and Williams-Beuren syndrome. This outcome represents a significant step towards better conceptualization of visual social world in DS and neurodevelopmental disorders in general. (shrink)

This study examined self-recognition processing in both the auditory and visual modalities by determining how comparable hearing a recording of one’s own voice was to seeing photograph of one’s own face. We also investigated whether the simultaneous presentation of auditory and visual self-stimuli would either facilitate or inhibit self-identification. Ninety-one participants completed reaction-time tasks of self-recognition when presented with their own faces, own voices, and combinations of the two. Reaction time and errors made when responding with (...) both the right and left hand were recorded to determine if there were lateralization effects on these tasks. Our findings showed that visual self-recognition for facial photographs appears to be superior to auditory self-recognition for voice recordings. Furthermore, a combined presentation of one’s own face and voice appeared to inhibit rather than facilitate self-recognition and there was a left-hand advantage for reaction time on the combined-presentation tasks. (shrink)

Visual Agnosia is a comprehensive and up-to-date review of disorders of higher vision that relates these disorders to current conceptions of higher vision from cognitive science, illuminating both the neuropsychological disorders and the nature of normal visual object recognition.Brain damage can lead to selective problems with visual perception, including visual agnosia the inability to recognize objects even though elementary visual functions remain unimpaired. Such disorders are relatively rare, yet they provide a window onto how (...) the normal brain might accomplish the complex task of vision. Visual Agnosia reviews a century of case studies of higher-level visual deficits following brain damage, places them in the general context of current neuroscience, and draws relevant conclusions about the organization of normal visual processing. It is unique in drawing on research in cognitive psychology, computational vision, visual neurophysiology, and neuropsychology to interpret the agnosias and draw inferences from them about visual object recognition.Following a historical account of agnosia research, Visual Agnosia offers a taxonomy of a wide range of agnosia syndromes, describing and interpreting the syndromes in terms of the latest theoretical models of visual processing and ultimately bringing them to bear as evidence on a variety of questions in the study of higher vision.Martha J. Farah is Associate Professor of Psychology at Carnegie Mellon University. Visual Agnosia is included in the Issues in Biology of Language and Cognition series, edited by John Marshall. (shrink)