Many organisms possess multiple sensory systems, such as vision, hearing, touch, smell, and taste. The possession of multiple ways of sensing the world offers many benefits. However, combining information from different senses also poses many challenges for the nervous system. In recent years there has been dramatic progress in understanding how information from the different senses gets integrated in order to construct useful representations of external space. This volume brings together the leading researchers from a broad range of scientific approaches (...) to present the first overview of this central topic in cognitive neuroscience. (shrink)
Following on from ecological theories of perception, such as the one proposed by [Gibson, J. J. . The senses considered as perceptual systems. Boston: Houghton Mifflin] this paper reviews the literature on the multisensory interactions underlying the perception of flavor in order to determine the extent to which it is really appropriate to consider flavor perception as a distinct perceptual system. We propose that the multisensory perception of flavor may be indicative of the fact that the taxonomy currently used to (...) define our senses is simply not appropriate. According to the view outlined here, the act of eating allows the different qualities of foodstuffs to be combined into unified percepts; and flavor can be used as a term to describe the combination of tastes, smells, trigeminal, and tactile sensations as well as the visual and auditory cues, that we perceive when tasting food. (shrink)
Is consciousness multisensory? Obviously it is multisensory in certain ways. Human beings typically possess the capacity to have experiences in at least the five familiar sensory modalities, and quite possibly in a number of other less commonly recognised modalities as well. But there are other respects in which it is far from obvious that consciousness is multisensory. This chapter is concerned with one such respect. Οur concern here is with whether consciousness contains experiences associated with distinct modalities at the same (...) time. We will describe those who endorse a positive answer to this question as endorsing a multisensory view (MSV) of the structure of consciousness, and those who endorse a negative answer to this question as having a unisensory view (USV) of the structure of consciousness. (shrink)
The law of prior entry was one of E.B. Titchener’s seven fundamental laws of attention. According to Titchener : “the object of attention comes to consciousness more quickly than the objects which we are not attending to.” Although researchers have been studying prior entry for more than a century now, progress in understanding the effect has been hindered by the many methodological confounds present in early research. As a consequence, it is unclear whether the behavioral effects reported in the majority (...) of published studies in this area should be attributed to attention, decisional response biases, and/or, in the case of exogenous spatial cuing studies of the prior-entry effect, to sensory facilitation effects instead. In this article, the literature on the prior-entry effect is reviewed, the confounds present in previous research highlighted, current consensus summarized, and some of the key questions for future research outlined. In particular, recent research has now provided compelling psychophysical and electrophysiological evidence to support the claim that attending to a sensory modality, spatial location, or stimulus feature/attribute can all give rise to a relative speeding-up of the time of arrival of attended, as compared to relatively less attended stimuli. Prior-entry effects have now been demonstrated following both the endogenous and exogenous orienting of attention, though prior-entry effects tend to be smaller in magnitude when assessed by means of participants’ performance on SJ tasks than when assessed by means of their performance on TOJ tasks. (shrink)
The last couple of years have seen a rapid growth of interest in the study of crossmodal correspondences – the tendency for our brains to preferentially associate certain features or dimensions of stimuli across the senses. By now, robust empirical evidence supports the existence of numerous crossmodal correspondences, affecting people’s performance across a wide range of psychological tasks – in everything from the redundant target effect paradigm through to studies of the Implicit Association Test, and from speeded discrimination/classification tasks through (...) to unspeeded spatial localisation and temporal order judgment tasks. However, one question that has yet to receive a satisfactory answer is whether crossmodal correspondences automatically affect people’s performance , as opposed to reflecting more of a strategic, or top-down, phenomenon. Here, we review the latest research on the topic of crossmodal correspondences to have addressed this issue. We argue that answering the question will require researchers to be more precise in terms of defining what exactly automaticity entails. Furthermore, one’s answer to the automaticity question may also hinge on the answer to a second question: Namely, whether crossmodal correspondences are all ‘of a kind’, or whether instead there may be several different kinds of crossmodal mapping . Different answers to the automaticity question may then be revealed depending on the type of correspondence under consideration. We make a number of suggestions for future research that might help to determine just how automatic crossmodal correspondences really are. (shrink)
Are two senses more certain than one? Subjective confidence, as an instance of metacognition, has mostly been investigated on a sense-by-sense basis. Yet perception is most frequently multisensory. Here we consider the implications and relevance of understanding confidence at the multisensory level.
People’s awareness of tactile stimuli has been investigated in far less detail than their awareness of stimuli in other sensory modalities. In an attempt to fill this gap, we provide an overview of studies that are pertinent to the topic of tactile consciousness. We discuss the results of research that has investigated phenomena such as “change blindness”, phantom limb sensations, and numerosity judgments in tactile perception, together with the results obtained from the study of patients affected by deficits that can (...) adversely affect tactile perception such as neglect, extinction, and numbsense. The similarities as well as some of the important differences that have emerged when visual and tactile conscious information processing have been compared using similar experimental procedures are highlighted. We suggest that conscious information processing in the tactile modality cannot be separated completely from the more general processing of spatial information in the brain. Finally, the importance of considering tactile consciousness within the larger framework of multisensory information processing is also discussed. (shrink)
The sounds that result from our movement and that mark the outcome of our actions typically convey useful information concerning the state of our body and its movement, as well as providing pertinent information about the stimuli with which we are interacting. Here we review the rapidly growing literature investigating the influence of non-veridical auditory cues (i.e., inaccurate in terms of their context, timing, and/or spectral distribution) on multisensory body and action perception, and on motor behavior. Inaccurate auditory cues provide (...) a unique opportunity to study cross-modal processes: the ability to detect the impact of each sense when they provide a slightly different message is greater. Additionally, given that similar cross-modal processes likely occur regardless of the accuracy or inaccuracy of sensory input, studying incongruent interactions are likely to also help us predict interactions between congruent inputs. The available research convincingly demonstrates that perceptions of the body, of movement, and of surface contact features (e.g., roughness) are influenced by the addition of non-veridical auditory cues. Moreover, auditory cues impact both motor behavior and emotional valence, the latter showing that sounds that are highly incongruent with the performed movement induce feelings of unpleasantness (perhaps associated with lower processing fluency). Such findings are relevant to the design of auditory cues associated with product interaction, and the use of auditory cues in sport performance and therapeutic situations given the impact on motor behavior. (shrink)
The last decade has seen great progress in the study of the nature of crossmodal links in exogenous and endogenous spatial attention . Exogenous spatial cuing studies of human crossmodal attention and multisensory integration. In C. Spence, & J. Driver , Crossmodal space and crossmodal attention . Oxford, UK: Oxford University Press.], for a recent review). A growing body of research now highlights the existence of robust crossmodal links between auditory, visual, and tactile spatial attention. However, until recently, studies of (...) exogenous and endogenous attention have proceeded relatively independently. In daily life, however, these two forms of attentional orienting continuously compete for the control of our attentional resources, and ultimately, our awareness. It is therefore critical to try and understand how exogenous and endogenous attention interact in both the unimodal context of the laboratory and the multisensory contexts that are more representative of everyday life. To date, progress in understanding the interaction between these two forms of orienting has primarily come from unimodal studies of visual attention. We therefore start by summarizing what has been learned from this large body of empirical research, before going on to review more recent studies that have started to investigate the interaction between endogenous and exogenous orienting in a multisensory setting. We also discuss the evidence suggesting that exogenous spatial orienting is not truly automatic, at least when assessed in a crossmodal context. Several possible models describing the interaction between endogenous and exogenous orienting are outlined and then evaluated in terms of the extant data. (shrink)
Color conveys critical information about the flavor of food and drink by providing clues as to edibility, flavor identity, and flavor intensity. Despite the fact that more than 100 published papers have investigated the influence of color on flavor perception in humans, surprisingly little research has considered how cognitive and contextual constraints may mediate color–flavor interactions. In this review, we argue that the discrepancies demonstrated in previously-published color–flavor studies may, at least in part, reflect differences in the sensory expectations that (...) different people generate as a result of their prior associative experiences. We propose that color–flavor interactions in flavor perception cannot be understood solely in terms of the principles of multisensory integration but that the role of higher-level cognitive factors, such as expectations, must also be considered. (shrink)
For more than a century now, researchers have acknowledged the existence of seemingly arbitrary crossmodal congruency effects between dimensions of sensory stimuli in the general population. Such phenomena, known by a variety of terms including 'crossmodal correspondences', involve individual stimulus properties, rely on a crossmodal mapping of unisensory features, and appear to be shared by the majority of individuals. In other words, members of the general population share underlying preferences for specific pairings across the senses. Crossmodal correspondences between complementary sensory (...) cues have often been referred to as synesthetic correspondences but, we would argue, differ from full-blown synesthetic experiences in a number of important ways, including the fact that there are no idiosyncratic concurrent sensations. Recent psychophysical evidence suggests that such crossmodal correspondences can modulate multisensory integration by helping to resolve the crossmodal binding problem. Here, we propose a model to account for the effects of crossmodal correspondences between complementary auditory and visual cues and critically review their relation to full-blown synesthesia. (shrink)
Change blindness, the surprising inability of people to detect significant changes between consecutively-presented visual displays, has recently been shown to affect tactile perception as well. Visual change blindness has been observed during saccades and eye blinks, conditions under which people’s awareness of visual information is temporarily suppressed. In the present study, we demonstrate change blindness for suprathreshold tactile stimuli resulting from the execution of a secondary task requiring bodily movement. In Experiment 1, the ability of participants to detect changes between (...) two sequentially-presented vibrotactile patterns delivered on their arms and legs was compared while they performed a secondary task consisting of either the execution of a movement with the right arm toward a visual target or the verbal identification of the target side. The results demonstrated that a motor response gave rise to the largest drop in perceptual sensitivity in detecting changes to the tactile display. In Experiment 2, we replicated these results under conditions in which the participants had to detect tactile changes while turning a steering wheel instead. These findings are discussed in terms of the role played by bodily movements, sensory suppression, and higher order information processing in modulating people’s awareness of tactile information across the body surface. (shrink)
Multisensory integration can alter information processing, and previous research has shown that such processes are modulated by sensory switch costs and prior experience. Here we report an incidental finding demonstrating, for the first time, the interplay between these processes and experimental factors, specifically the presence of the experimenter in the testing room. Experiment 1 demonstrates that multisensory motor facilitation in response to audiovisual stimuli is higher in those trials in which the sensory modality switches than when it repeats. Those participants (...) who completed the study while alone exhibited increased RT variability. Experiment 2 replicated these findings using the letters “b” and “d” presented as unisensory stimuli or congruent and incongruent multisensory stimuli. Multisensory enhancements were inflated following a sensory switch; that is, congruent and incongruent multisensory stimuli resulted in significant gains following a sensory switch in the monitored condition. However, when the participants were left alone, multisensory enhancements were only observed for repeating incongruent multisensory stimuli. These incidental findings therefore suggest that the effects of letter congruence and sensory switching on multisensory integration are partly modulated by the presence of an experimenter. (shrink)
Dijkerman & de Haan (D&dH) argue that body image and body schema form parts of different and dissociable somatosensory streams. We agree in general, but believe that more emphasis should be placed on interactions between these two streams. We illustrate this point with evidence from the rubber-hand illusion (RHI) – an illusion of body image, which depends critically upon body schema.
Bistable figures provide a fascinating window through which to explore human visual awareness. Here we demonstrate for the first time that the semantic context provided by a background auditory soundtrack can modulate an observer’s predominant percept while watching the bistable “my wife or my mother-in-law” figure . The possibility of a response-bias account—that participants simply reported the percept that happened to be congruent with the soundtrack that they were listening to—was excluded in Experiment 2. We further demonstrate that this crossmodal (...) semantic effect was additive with the manipulation of participants’ visual fixation , while it interacted with participants’ voluntary attention . These results indicate that audiovisual semantic congruency constrains the visual processing that gives rise to the conscious perception of bistable visual figures. Crossmodal semantic context therefore provides an important mechanism contributing to the emergence of visual awareness. (shrink)
People can maintain accurate representations of visual changes without necessarily being aware of them. Here, we investigate whether a similar phenomenon also exists in touch. In Experiments 1 and 2, participants detected the presence of a change between two consecutively-presented tactile displays. Tactile change blindness was observed, with participants failing to report the presence of tactile change. Critically, however, when participants had to make a forced choice response regarding the number of stimuli presented in the two displays, their performance was (...) significantly better than chance . Experiment 3 demonstrated that tactile change detection does not necessarily involve a shift of spatial attention toward the location of change, regardless of whether the change is explicitly detected. We conclude that tactile change detection likely results from comparing representations of the two displays, rather than by directing spatial attention to the location of the change. (shrink)
Mareschal and his colleagues argue that cognition consists of partial representations emerging from organismic constraints placed on information processing through development. However, any notion of constraints must consider multiple sensory modalities, and their gradual integration across development. Multisensory integration constitutes one important way in which developmental constraints may lead to enriched representations that serve more than immediate behavioural goals.
This collection of essays brings together research on sense modalities in general and spatial perception in particular in a systematic and interdisciplinary way. It updates a long-standing philosophical fascination with this topic by incorporating theoretical and empirical research from cognitive science, neuroscience, and psychology. The book is divided thematically to cover a wide range of established and emerging issues. Part I covers notions of objectivity and subjectivity in spatial perception and thinking. Part II focuses on the canonical distal senses, such (...) as vision and audition. Part III concerns the chemical senses, including olfaction and gustation. Part IV discusses bodily awareness, peripersonal space, and touch. Finally, the volume concludes with Part V, on multimodality. Spatial Senses is an important contribution to the scholarly literature on the philosophy of perception that takes into account important advances in the sciences. (shrink)
The ability to resist distracting stimuli whilst voluntarily focusing on a task is fundamental to our everyday cognitive functioning. Here, we investigated how this ability develops, and thereafter declines, across the lifespan using a single task/experiment. Young children (5–7 years), older children (10–11 years), young adults (20–27 years), and older adults (62–86 years) were presented with complex visual scenes. Endogenous (voluntary) attention was engaged by having the participants search for a visual target presented on either the left or right side (...) of the display. The onset of the visual scenes was preceded – at stimulus onset asynchronies (SOAs) of 50, 200, or 500 ms – by a task-irrelevant sound (an exogenous crossmodal spatial distractor) delivered either on the same or opposite side as the visual target, or simultaneously on both sides (cued, uncued, or neutral trials, respectively). Age-related differences were revealed, especially in the extreme age-groups, which showed a greater impact of crossmodal spatial distractors. Young children were highly susceptible to exogenous spatial distraction at the shortest SOA (50 ms), whereas older adults were distracted at all SOAs, showing significant exogenous capture effects during the visual search task. By contrast, older children and young adults' search performance was not significantly affected by crossmodal spatial distraction. Overall, these findings present a detailed picture of the developmental trajectory of endogenous resistance to crossmodal spatial distraction from childhood to old age and demonstrate a different efficiency in coping with distraction across the four age-groups studied. (shrink)