The sounds that result from our movement and that mark the outcome of our actions typically convey useful information concerning the state of our body and its movement, as well as providing pertinent information about the stimuli with which we are interacting. Here we review the rapidly growing literature investigating the influence of non-veridical auditory cues (i.e., inaccurate in terms of their context, timing, and/or spectral distribution) on multisensory body and action perception, and on motor behavior. Inaccurate auditory (...) cues provide a unique opportunity to study cross-modal processes: the ability to detect the impact of each sense when they provide a slightly different message is greater. Additionally, given that similar cross-modal processes likely occur regardless of the accuracy or inaccuracy of sensory input, studying incongruent interactions are likely to also help us predict interactions between congruent inputs. The available research convincingly demonstrates that perceptions of the body, of movement, and of surface contact features (e.g., roughness) are influenced by the addition of non-veridical auditory cues. Moreover, auditory cues impact both motor behavior and emotional valence, the latter showing that sounds that are highly incongruent with the performed movement induce feelings of unpleasantness (perhaps associated with lower processing fluency). Such findings are relevant to the design of auditory cues associated with product interaction, and the use of auditory cues in sport performance and therapeutic situations given the impact on motor behavior. (shrink)

This study aimed to explore the psychophysical bases of multisensory surface stickiness perception by investigating how sensitively humans perceive different levels of stickiness intensity conveyed by auditory, tactile, and visual cues. First, we sorted five different sticky stimuli by perceived intensity in ascending order for each modality separately and evaluated the discrimination sensitivities of each participant using a fitted psychometric curve. Results showed that perceptual intensity orders were not identical to physical intensity order and that the sequential order of (...) perceived intensities for different modalities was inconsistent. Moreover, estimated perceptual sensitivities to surface stickiness indicated that auditory cues result in better discrimination sensitivity than tactile and visual cues. Second, we calculated the relative perceptual distances of stickiness intensities using multidimensional scaling. A follow-up statistical test demonstrated that the perceptual mapping of vision and touch are similar but that auditory perception is different. These results suggest that the discriminability of stickiness intensity is best served by auditory cues and that texture information processing in the auditory domain is distinctive from that of other modalities. (shrink)

Speech unfolds over time, and the cues for even a single phoneme are rarely available simultaneously. Consequently, to recognize a single phoneme, listeners must integrate material over several hundred milliseconds. Prior work contrasts two accounts: (a) a memory buffer account in which listeners accumulate auditory information in memory and only access higher level representations (i.e., lexical representations) when sufficient information has arrived; and (b) an immediate integration scheme in which lexical representations can be partially activated on the basis of (...) early cues and then updated when more information arises. These studies have uniformly shown evidence for immediate integration for a variety of phonetic distinctions. We attempted to extend this to fricatives, a class of speech sounds which requires not only temporal integration of asynchronous cues (the frication, followed by the formant transitions 150–350 ms later), but also integration across different frequency bands and compensation for contextual factors like coarticulation. Eye movements in the visual world paradigm showed clear evidence for a memory buffer. Results were replicated in five experiments, ruling out methodological factors and tying the release of the buffer to the onset of the vowel. These findings support a general auditory account for speech by suggesting that the acoustic nature of particular speech sounds may have large effects on how they are processed. It also has major implications for theories of auditory and speech perception by raising the possibility of an encapsulated memory buffer in early auditory processing. (shrink)

Speech unfolds over time, and the cues for even a single phoneme are rarely available simultaneously. Consequently, to recognize a single phoneme, listeners must integrate material over several hundred milliseconds. Prior work contrasts two accounts: (a) a memory buffer account in which listeners accumulate auditory information in memory and only access higher level representations (i.e., lexical representations) when sufficient information has arrived; and (b) an immediate integration scheme in which lexical representations can be partially activated on the basis of (...) early cues and then updated when more information arises. These studies have uniformly shown evidence for immediate integration for a variety of phonetic distinctions. We attempted to extend this to fricatives, a class of speech sounds which requires not only temporal integration of asynchronous cues (the frication, followed by the formant transitions 150–350 ms later), but also integration across different frequency bands and compensation for contextual factors like coarticulation. Eye movements in the visual world paradigm showed clear evidence for a memory buffer. Results were replicated in five experiments, ruling out methodological factors and tying the release of the buffer to the onset of the vowel. These findings support a general auditory account for speech by suggesting that the acoustic nature of particular speech sounds may have large effects on how they are processed. It also has major implications for theories of auditory and speech perception by raising the possibility of an encapsulated memory buffer in early auditory processing. (shrink)

The auditory gaze cueing effect (auditory-GCE) is a faster response to auditory targets at an eye-gaze cue location than at a non-cue location. Previous research has found that auditory-GCE can be influenced by the integration of both gaze direction and emotion conveyed through facial expressions. However, it is unclear whether the emotional information of auditory targets can be cross-modally integrated with gaze direction to affect auditory-GCE. Here, we set neutral faces with different gaze directions (...) as cues and three emotional sounds (fearful, happy, and neutral) as targets to investigate how the emotion of sound target modulates the auditory-GCE. Moreover, we conducted a controlled experiment using arrow cues. The results show that the emotional content of sound targets influences the auditory-GCE but only for those induced by facial cues. Specifically, fearful sounds elicit a significantly larger auditory-GCE compared to happy and neutral sounds, indicating that the emotional content of auditory targets plays a modulating role in the auditory-GCE. Furthermore, this modulation appears to occur only at a higher level of social meaning, involving the integration of emotional information from a sound with social gaze direction, rather than at a lower level, which involves the integration of direction and auditory emotion. (shrink)

Recently, researchers have expanded the investigation into attentional biases toward positive stimuli; however, few studies have examined attentional biases toward positive auditory information. In three experiments, the present study employed an emotional spatial cueing task using emotional sounds as cues and auditory stimuli or visual stimuli as targets to explore whether auditory or visual spatial attention could be modulated by positive auditory cues. Experiment 3 also examined the temporal dynamics of cross-modal auditory bias toward positive (...) natural sounds using event-related potentials. The behavioral results of the three experiments consistently demonstrated that response times to targets were faster after positive auditory cues than they were after neutral auditory cues in the valid condition, indicating that healthy participants showed a selective auditory attentional bias and cross-modal attentional bias toward positive natural sounds. The results of Experiment 3 showed that N1 amplitudes were more negative after positive sounds than they were after neutral sounds, which further provided electrophysiological evidence that positive auditory information enhances attention at early stages in healthy adults. The results of the experiments performed in the present study suggest that humans exhibit an attentional bias toward positive natural sounds. (shrink)

It might be suggested that in auditory experience elements of the material world are not apparent to us in the way they are in vision and touch, and that this constitutes a shortcoming in the kind of cognitive contact with the world provided by auditory perception. I develop this suggestion, and then set out a way of thinking about the appearances of sound‐producing events that might provide a response.1Thanks to James Stazicker for very helpful comments and suggestions.

Multisensory integration allows us to link sensory cues from multiple sources and plays a crucial role in speech development. However, it is not clear whether humans have an innate ability or whether repeated sensory input while the brain is maturing leads to efficient integration of sensory information in speech. We investigated the integration of auditory and somatosensory information in speech processing in a bimodal perceptual task in 15 young adults (age 19 to 30) and 14 children (age 5 to (...) 6). The participants were asked to identify if the perceived target was the sound /e/ or /ø/. Half of the stimuli were presented under a unimodal condition with only auditory input. The other stimuli were presented under a bimodal condition with both auditory input and somatosensory input consisting of facial skin stretches provided by a robotic device, which mimics the articulation of the vowel /e/. The results indicate that the effect of somatosensory information on sound categorization was larger in adults than in children. This suggests that integration of auditory and somatosensory information evolves throughout the course of development. (shrink)

This study investigated eye-movement patterns during emotion perception for children with hearing aids and hearing children. Seventy-eight participants aged from 3 to 7 were asked to watch videos with a facial expression followed by an oral statement, and these two cues were either consistent or inconsistent in emotional valence. Results showed that while normal-hearing children paid more attention to the upper part of the face, children with hearing aids paid more attention to the lower face after the oral statement was (...) presented. When there was an inconsistency between the visual and auditory cues, children with hearing aids were likely to increase attention to the upper face. The results revealed the underlying mechanism of deficits in emotion perception for hearing-impaired children. (shrink)

Our understanding of body ownership largely relies on the Rubber Hand Illusion (RHI) paradigm where synchronous stroking of the real and fake hands leads to an illusion of ownership of RH provided its physical, anatomical, and spatial plausibility. Self-attribution of a fake hand also occurs during visuomotor synchrony, when the visual feedback of self-initiated movements follows the trajectory of the instantiated motor command. In both cases, the experience of ownership is established through bottom-up integration and top-down prediction of multisensory (proximodistal) (...) cues within the peripersonal space. It seems, however, that depending on whether the signals are externally or self-generated, top-down expectation signals are qualitatively different. On the one hand, in RHI, the sensory correlations are modulated by empirically induced priors about the internal model of the body. On the other hand, in mRHI, body ownership is actively shaped by processes allowing for continuous comparison between the actual and predicted sensory consequences of the actions. Ample research demonstrates that the differential processing of the predicted and the actual reafferent information is addressed by the central nervous system via an internal Forward Model (FM). Indeed, results from mRHI suggest that in action-contexts mechanism underlying body ownership could be similar to the FM. Crucially FMs integrate across all sensory input including not only proximodistal (visuomotor) but also purely distal sensory signals (visuoauditory). Thus if body ownership results from a consistency of FM, it would be affected by the (in)congruency of purely distal signals provided that they inform about the action-consequences relevant to the task. Here we explicitly address this question. To test our hypothesis, we devise an embodied VR-based task where action outcomes were signaled by distinct auditory cues. By manipulating the cues with respect to their spatial, temporal and semantic congruency, we show that purely distal feedback violating predictions about action outcomes compromises both body ownership and performance. These results demonstrate that body ownership is influenced by cues which not only pertain to the body itself but also those arising outside of the body, and suggest that in goal-oriented tasks body ownership might result from the consistency of forward models. (shrink)

Self–other discrimination was investigated with point-light displays in which actions were presented with or without additional auditory information. Participants first executed different actions in time with music. In two subsequent experiments, they watched point-light displays of their own or another participant’s recorded actions, and were asked to identify the agent . Manipulations were applied to the visual information and to the auditory information . Results indicate that self-recognition was better than chance in all conditions and was highest when (...) observing relatively unconstrained patterns of movement. Auditory information did not increase accuracy even with the most ambiguous visual displays, suggesting that judgments of agent identity depend much more on motor cues than on auditory or audiovisual information. (shrink)

Many situations require the simultaneous processing of auditory and visual information, however, stimuli presented to one sensory modality can sometimes interfere with processing in a second sensory modality (i.e., modality dominance). The current study further investigated modality dominance by examining how task demands and bimodal presentation affect speeded auditory and visual discriminations. Participants in the current study had to quickly determine if two words, two pictures, or two word-picture pairings were the same or different, and we manipulated task (...) demands across three different conditions. In an immediate recognition task, there was only one second between the two stimuli/stimulus pairs and auditory dominance was found. Compared to the respective unimodal baselines, pairing pictures and words together slowed down visual responses and sped up auditory responses. Increasing the interstimulus interval to four seconds and blocking verbal rehearsal weakened auditory dominance effects, however, conflicting and redundant visual cues sped up auditory discriminations. Thus, simultaneously presenting pictures and words had different effects on auditory and visual processing, with bimodal presentation slowing down visual processing and speeding up auditory processing. These findings are consistent with a proposed mechanism underlying auditory dominance, which posits that auditory stimuli automatically grab attention and attenuate/delay visual processing. (shrink)

Three experiments examined repetition priming for meaningful environmental sounds in a sound stem identification paradigm using brief sound cues. Prior encoding of target sounds together with their associated names facilitated subsequent identification of sound stems relative to nonstudied controls. In contrast, prior exposure to names alone in the absence of the environmental sounds did not prime subsequent sound stem identification performance at all . Explicit and implicit memory were dissociated such that sound stem cued recall was higher following semantic than (...) nonsemantic encoding, whereas sound priming was insensitive to manipulations of depth encoding . These results extend the findings of long-term repetition priming into the auditory nonverbal domain and suggest that priming for environmental sounds is mediated primarily by perceptual processes. (shrink)

Previous research in the visual domain suggests that exogenous attention in form of peripheral cueing increases spatial but lowers temporal resolution. It is unclear whether this effect transfers to other sensory modalities. Here, we tested the effects of exogenous attention on temporal and spectral resolution in the auditory domain. Eighteen young, normal-hearing adults were tested in both gap and frequency change detection tasks with exogenous cuing. Benefits of valid cuing were only present in the gap detection task while costs (...) of invalid cuing were observed in both tasks. Our results suggest that exogenous attention in the auditory system improves temporal resolution without compromising spectral resolution. (shrink)

We examined whether facial expressions of performers influence the emotional connotations of sung materials, and whether attention is implicated in audio-visual integration of affective cues. In Experiment 1, participants judged the emotional valence of audio-visual presentations of sung intervals. Performances were edited such that auditory and visual information conveyed congruent or incongruent affective connotations. In the single-task condition, participants judged the emotional connotation of sung intervals. In the dual-task condition, participants judged the emotional connotation of intervals while performing a (...) secondary task. Judgements were influenced by melodic cues and facial expressions and the effects were undiminished by the secondary task. Experiment 2 involved identical conditions but participants were instructed to base judgements on auditory information alone. Again, facial expressions influenced judgements and the effect was undiminished by the secondary task. The results suggest that visual aspects of music performance are automatically and preattentively registered and integrated with auditory cues. (shrink)

Religion can have an important influence in moral decision-making, and religious reminders may deter people from unethical behavior. Previous research indicated that religious contexts may increase prosocial behavior and reduce cheating. However, the perceptual-behavioral link between religious contexts and decision-making lacks thorough scientific understanding. This study adds to the current literature by testing the effects of purely audial religious symbols (instrumental music) on moral behavior across three different sites: Mauritius, the Czech Republic, and the USA. Participants were exposed to one (...) of three kinds of auditory stimuli (religious, secular, or white noise), and subsequently were given a chance to dishonestly report on solved mathematical equations in order to increase their monetary reward. The results showed cross-cultural differences in the effects of religious music on moral behavior, as well as a significant interaction between condition and religiosity across all sites, suggesting that religious participants were more influenced by the auditory religious stimuli than non-religious participants. We propose that religious music can function as a subtle cue associated with moral standards via cultural socialization and ritual participation. Such associative learning can charge music with specific meanings and create sacred cues that influence normative behavior. Our findings provide preliminary support for this view, which we hope further research will investigate more closely. (shrink)

Perceptual grouping is fundamental to many auditory processes. The Iambic–Trochaic Law (ITL) is a default grouping strategy, where rhythmic alternations of duration are perceived iambically (weak‐strong), while alternations of intensity are perceived trochaically (strong‐weak). Some argue that the ITL is experience dependent. For instance, French speakers follow the ITL, but not as consistently as German speakers. We hypothesized that learning about prosodic patterns, like word stress, modulates this rhythmic grouping. We tested this idea by training French adults on a (...) German‐like stress contrast. Individuals who showed better phonological learning had more ITL‐like grouping, particularly over duration cues. In a non‐phonological condition, French adults were trained using identical stimuli, but they learned to attend to acoustic variation that was not linguistic. Here, no learning effects were observed. Results thus suggest that phonological learning can modulate low‐level auditory grouping phenomena, but it is constrained by the ability of individuals to learn from short‐term training. (shrink)

Mueller et al. discuss the role of acoustic cues in detecting language structure more generally. Across languages, there are clear links between acoustic cues and syntactic structure. They show that AGL experiments implementing analogous links demonstrate that prosodic cues, as well as various auditory biases, facilitate the learning of structural rules. Some of these biases, e.g. for auditory grouping, are also present in other species.

Rhythm perception involves strong auditory-motor connections that can be enhanced with movement. However, it is unclear whether just seeing someone moving to a rhythm can enhance auditory-motor coupling, resulting in stronger entrainment. Rhythmic priming studies show that presenting regular rhythms before naturally spoken sentences can enhance grammaticality judgments compared to irregular rhythms or other baseline conditions. The current study investigated whether introducing a point-light figure moving in time with regular rhythms could enhance the rhythmic priming effect. Three experiments (...) revealed that the addition of a visual cue did not benefit rhythmic priming in comparison to auditory conditions with a static image. In Experiment 1, grammaticality judgments were poorer after audio-visual regular rhythms compared to auditory-only regular rhythms. In Experiments 2 and 3, there was no difference in grammaticality judgments after audio-visual regular rhythms compared to auditory-only irregular rhythms for either a bouncing point-light figure or a swaying point-light figure. Comparison of the observed performance with previous data suggested that the audio-visual component removed the regular prime benefit. These findings suggest that the visual cues used in this study do not enhance rhythmic priming and could hinder the effect by potentially creating a dual-task situation. In addition, individual differences in sensory-motor and social scales of music reward influenced the effect of the visual cue. Implications for future audio-visual experiments aiming to enhance beat processing, and the importance of individual differences will be discussed. (shrink)

Animal vocalizations convey important information about the emitter, including sex, age, biological quality, and emotional state. Early on, Darwin proposed that sex differences in auditory signals and vocalizations were driven by sexual selection mechanisms. In humans, studies on the association between male voice attributes and physical formidability have thus far reported mixed results. Hence, with a view to furthering our understanding of the role of human voice in advertising physical formidability, we sought to identify acoustic attributes of male voices (...) associated with physical formidability proxies. Mean fundamental frequency, formant dispersion, formant position, and vocal tract length data from a sample of 101 male voices was analyzed for potential associations with height, weight, and maximal handgrip strength. F0 correlated negatively with HGS; Pf showed negative correlations with HGS, height and weight, whereas VTL positively correlated with HGS, height and weight. All zero-order correlations remained significant after controlling for false discovery rate with the Benjamini–Hochberg method. After controlling for height and weight—and controlling for FDR—the correlation between F0 and HGS remained significant. In addition, to evaluate the ability of human male voices to advertise physical formidability to potential mates, 151 heterosexual female participants rated the voices of the 10 strongest and the 10 weakest males from the original sample for perceived physical strength, and given that physical strength is a desirable attribute in male partners, perceived attractiveness. Generalized linear mixed model analyses—which allow for generalization of inferences to other samples of both raters and targets—failed to support a significant association of perceived strength or attractiveness from voices alone and actual physical strength. These results add to the growing body of work on the role of human voices in conveying relevant biological information. (shrink)

What are the factors that contribute to (or inhibit) diachronic sound change? While acoustically motivated sound changes are well documented, research on the articulatory and audiovisual-perceptual aspects of sound change is limited. This paper investigates the interaction of articulatory variation and audiovisual speech perception in the Northern Cities Vowel Shift (NCVS), a pattern of sound change observed in the Great Lakes region of the United States. We focus specifically on the maintenance of the contrast between the vowels /ɑ/ and /ɔ/, (...) both of which are fronted as a result of the NCVS. We present results from two experiments designed to test how the NCVS is produced and perceived. In the first experiment, we present data from an articulatory and acoustic analysis of the production of fronted /ɑ/ and /ɔ/. We find that some speakers distinguish /ɔ/ from /ɑ/ with a combination of both tongue position and lip rounding, while others do so using either tongue position or lip rounding alone. For speakers who distinguish /ɔ/ from /ɑ/ along only one articulatory dimension, /ɑ/ and /ɔ/ are acoustically more similar than for speakers who produce multiple articulatory distinctions. While all three groups of speakers maintain some degree of acoustic contrast between the vowels, the question is raised as to whether these articulatory strategies differ in their perceptibility. In the perception experiment, we test the hypothesis that visual speech cues play a role in maintaining contrast between the two sounds. The results of this experiment suggest that articulatory configurations in which /ɔ/ is produced with unround lips are perceptually weaker than those in which /ɔ/ is produced with rounding, even though these configurations result in acoustically similar output. We argue that these findings have implications for theories of sound change and variation in at least two respects: 1) visual cues can shape phonological systems through misperception-based sound change, and 2) phonological systems may be optimized not only for auditory but also for visual perceptibility. (shrink)

Social-rank cues communicate social status or social power within and between groups. Information about social-rank is fluently processed in both visual and auditory modalities. So far, the investigation on the processing of social-rank cues has been limited to studies in which information from a single modality was assessed or manipulated. Yet, in everyday communication, multiple information channels are used to express and understand social-rank. We sought to examine the voluntary nature of processing of facial and vocal signals of social-rank (...) using a cross-modal Stroop task. In two experiments, participants were presented with face-voice pairs that were either congruent or incongruent in social-rank. Participants’ task was to label face social dominance while ignoring the voice, or label voice social dominance while ignoring the face. In both experiments, we found that face-voice incongruent stimuli were processed more slowly and less accurately than were the congruent stimuli in the face-attend and the voice-attend tasks, exhibiting classical Stroop-like effects. These findings are consistent with the functioning of a social-rank bio-behavioural system which consistently and automatically monitors one’s social standing in relation to others and uses that information to guide behaviour. (shrink)

Despite the recognized importance of bodily movements in spatial audition, few studies have integrated action-based protocols with spatial hearing in the peripersonal space. Recent work shows that tactile feedback and active exploration allow participants to improve performance in auditory distance perception tasks. However, the role of the different aspects involved in the learning phase, such as voluntary control of movement, proprioceptive cues, and the possibility of self-correcting errors, is still unclear. We study the effect of guided reaching exploration on (...) perceptual learning of auditory distance in peripersonal space. We implemented a pretest-posttest experimental design in which blindfolded participants must reach for a sound source located in this region. They were divided into three groups that were differentiated by the intermediate training phase: Guided, an experimenter guides the participant’s arm to contact the sound source; Active, the participant freely explores the space until contacting the source; and Control, without tactile feedback. The effects of exploration feedback on auditory distance perception in the peripersonal space are heterogeneous. Both the Guided and Active groups change their performance. However, participants in the Guided group tended to overestimate distances more than those in the Active group. The response error of the Guided group corresponds to a generalized calibration criterion over the entire range of reachable distances. Whereas the Active group made different adjustments for proximal and distal positions. The results suggest that guided exploration can induce changes on the boundary of the auditory reachable space. We postulate that aspects of agency such as initiation, control, and monitoring of movement, assume different degrees of involvement in both guided and active tasks, reinforcing a non-binary approach to the question of activity-passivity in perceptual learning and supporting a complex view of the phenomena involved in action-based learning. (shrink)

Three experiments investigated the nature of visuo-auditory crossmodal cueing in a triadic setting: participants had to detect an auditory signal while observing another agent’s head facing one of the two laterally positioned auditory sources. Experiment 1 showed that when the agent’s eyes were open, sounds originating on the side of the agent’s gaze were detected faster than sounds originating on the side of the agent’s visible ear; when the agent’s eyes were closed this pat-tern of responses was (...) reversed. Two additional experiments showed that the results were sensitive to whether participants could infer a hearing function on the part of the agent. When no ear was depicted on the agent, only a gaze-side advantage was observed , but when the agent’s ear was covered , an ear side advantage was observed only when hearing could still be inferred but not when hearing was inferred to be diminished . The findings are discussed in the context of inferential and simulation processes and joint attention mechanisms. (shrink)

Human listeners are assumed to apply different strategies to improve speech recognition in background noise. Young listeners with normal hearing, e.g., have been shown to follow the voice of a particular speaker based on the fundamental and formant frequencies, which are both influenced by the gender, age, and size of the speaker. However, the auditory and cognitive processes that underlie the extraction and discrimination of these voice cues across speakers may be subject to age-related decline. The present study aimed (...) to examine the utilization of F0 and formant cues for voice discrimination in older adults with hearing expected for their age. Difference limens for VD were estimated in 15 healthy older adults and 35 young adults using only F0 cues, only formant frequency cues, and a combination of F0 + formant frequencies. A three-alternative forced-choice paradigm with an adaptive-tracking threshold-seeking procedure was used. Wechsler backward digit span test was used as a measure of auditory working memory. Trail Making Test was used to provide cognitive information reflecting a combined effect of processing speed, mental flexibility, and executive control abilities. The results showed that the mean VD thresholds of the older adults were poorer than those of the young adults for all voice cues, although larger variability was observed among the older listeners; both age groups found the formant cues more beneficial for VD, compared to the F0 cues, and the combined cues resulted in better thresholds, compared to each cue separately; significant associations were found for the older adults in the combined F0 + formant condition between VD and TMT scores, and between VD and hearing sensitivity, supporting the notion that a decline with age in both top-down and bottom-up mechanisms may hamper the ability of older adults to discriminate between voices. The present findings suggest that older listeners may have difficulty following the voice of a specific speaker and thus implementing doing so as a strategy for listening amid noise. This may contribute to understanding their reported difficulty listening in adverse conditions. (shrink)

Listeners can attend to and track instruments or singing voices in complex musical mixtures, even though the acoustical energy of sounds from individual instruments may overlap in time and frequency. In popular music, lead vocals are often accompanied by sound mixtures from a variety of instruments, such as drums, bass, keyboards, and guitars. However, little is known about how the perceptual organization of such musical scenes is affected by selective attention, and which acoustic features play the most important role. To (...) investigate these questions, we explored the role of auditory attention in a realistic musical scenario. We conducted three online experiments in which participants detected single cued instruments or voices in multi-track musical mixtures. Stimuli consisted of 2-s multi-track excerpts of popular music. In one condition, the target cue preceded the mixture, allowing listeners to selectively attend to the target. In another condition, the target was presented after the mixture, requiring a more “global” mode of listening. Performance differences between these two conditions were interpreted as effects of selective attention. In Experiment 1, results showed that detection performance was generally dependent on the target’s instrument category, but listeners were more accurate when the target was presented prior to the mixture rather than the opposite. Lead vocals appeared to be nearly unaffected by this change in presentation order and achieved the highest accuracy compared with the other instruments, which suggested a particular salience of vocal signals in musical mixtures. In Experiment 2, filtering was used to avoid potential spectral masking of target sounds. Although detection accuracy increased for all instruments, a similar pattern of results was observed regarding the instrument-specific differences between presentation orders. In Experiment 3, adjusting the sound level differences between the targets reduced the effect of presentation order, but did not affect the differences between instruments. While both acoustic manipulations facilitated the detection of targets, vocal signals remained particularly salient, which suggest that the manipulated features did not contribute to vocal salience. These findings demonstrate that lead vocals serve as robust attractor points of auditory attention regardless of the manipulation of low-level acoustical cues. (shrink)

The sense of agency is the perception of willfully causing something to happen. Wegner and Wheatley proposed three prerequisites for SA: temporal contiguity between an action and its effect, congruence between predicted and observed effects, and exclusivity . We investigated how temporal contiguity, congruence, and the order of two human agents’ actions influenced SA on a task where participants rated feelings of self-agency for producing a tone. SA decreased when tone onsets were delayed, supporting contiguity as important, but the order (...) of the agents’ actions also mattered. Relative contiguity was the main determinant of SA, as delayed tones were usually attributed to the most recent action. This was unaffected by contingencies between the two actors’ actions , showing that contiguity has a powerful influence on SA, even during joint action in the presence of other cues. (shrink)

The participants in the present study had to make speeded elevation discrimination responses to visual targets presented to the left or right of central fixation following the presentation of a task-irrelevant auditory cue on either the same or opposite side. In Experiment 1, the cues were presented from in front of the participants (from the same azimuthal positions as the visual targets). A standard crossmodal exogenous spatial cuing effect was observed, with participants responding significantly faster in the elevation discrimination (...) task to visual targets when both the auditory cues and the visual targets were presented on the same side. Experiment 2 replicated the exogenous spatial cuing effect for frontal visual targets following both front and rear auditory cues. The results of Experiment 3 demonstrated that the participants had little difficulty in correctly discriminating the location from which the sounds were presented. Thus, taken together, the results of the three experiments reported here demonstrate that the exact co-location of auditory cues and visual targets is not necessary to attract spatial attention. Implications of these results for the design of real-world warning signals are discussed. (shrink)

This review provides an overview of the research on nonverbal expressions of positive emotions, organised into emotion families, that is, clusters sharing common characteristics. Epistemological positive emotions are found to have distinct, recognisable displays via vocal or facial cues, while the agency-approach positive emotions appear to be associated with recognisable visual, but not auditory, cues. Evidence is less strong for the prosocial emotions in any modality other than touch, and there is little support for distinct recognisable signals of the (...) savouring positive emotions. In closing, some limitations of extant work are noted and some proposals for future research are outlined. (shrink)

Following on from ecological theories of perception, such as the one proposed by [Gibson, J. J. . The senses considered as perceptual systems. Boston: Houghton Mifflin] this paper reviews the literature on the multisensory interactions underlying the perception of flavor in order to determine the extent to which it is really appropriate to consider flavor perception as a distinct perceptual system. We propose that the multisensory perception of flavor may be indicative of the fact that the taxonomy currently used to (...) define our senses is simply not appropriate. According to the view outlined here, the act of eating allows the different qualities of foodstuffs to be combined into unified percepts; and flavor can be used as a term to describe the combination of tastes, smells, trigeminal, and tactile sensations as well as the visual and auditory cues, that we perceive when tasting food. (shrink)

Visual information conveyed by iconic hand gestures and visible speech can enhance speech comprehension under adverse listening conditions for both native and non‐native listeners. However, how a listener allocates visual attention to these articulators during speech comprehension is unknown. We used eye‐tracking to investigate whether and how native and highly proficient non‐native listeners of Dutch allocated overt eye gaze to visible speech and gestures during clear and degraded speech comprehension. Participants watched video clips of an actress uttering a clear or (...) degraded (6‐band noise‐vocoded) action verb while performing a gesture or not, and were asked to indicate the word they heard in a cued‐recall task. Gestural enhancement was the largest (i.e., a relative reduction in reaction time cost) when speech was degraded for all listeners, but it was stronger for native listeners. Both native and non‐native listeners mostly gazed at the face during comprehension, but non‐native listeners gazed more often at gestures than native listeners. However, only native but not non‐native listeners' gaze allocation to gestures predicted gestural benefit during degraded speech comprehension. We conclude that non‐native listeners might gaze at gesture more as it might be more challenging for non‐native listeners to resolve the degraded auditory cues and couple those cues to phonological information that is conveyed by visible speech. This diminished phonological knowledge might hinder the use of semantic information that is conveyed by gestures for non‐native compared to native listeners. Our results demonstrate that the degree of language experience impacts overt visual attention to visual articulators, resulting in different visual benefits for native versus non‐native listeners. (shrink)

Our research aimed to investigate the effectiveness of auditory, visual, and audiovisual warning signals for capturing the attention of the pilot, and how stimulus onset asynchronies in audiovisual stimuli affect pilots perceiving the bimodal warning signals under different perceptual load conditions. In experiment 1 of the low perceptual load condition, participants discriminated the location of visual targets preceded by five different types of warning signals. In experiment 2 of high perceptual load, participants completed the location task identical to a (...) low load condition and a digit detection task in a rapid serial visual presentation stream. The main effect of warning signals in two experiments showed that visual and auditory cues presented simultaneously could effectively and efficiently arouse the attention of the pilots in high and low load conditions. Specifically, auditory, AV, and visual preceding auditory stimulus by 100 ms increased the spatial orientation to a valid position in low load conditions. With the increase in visual perceptual load, auditory preceding the visual stimulus by 100 ms and A warning signals had stronger spatial orientation. The results are expected to theoretically support the optimization design of the cockpit display interface, contributing to immediate flight crew awareness. (shrink)

Two experiments investigated the role that mental set plays in reading aloud using the task choice procedure developed by Besner and Care [Besner, D., & Care, S. . A paradigm for exploring what the mind does while deciding what it should do. Canadian Journal of Experimental Psychology, 57, 311–320]. Subjects were presented with a word, and asked to either read it aloud or decide whether it appeared in upper/lower case. Task information, in the form of a brief auditory cue, (...) appeared 750 ms before the word, or at the same time as the word. Experiment 1 yielded evidence consistent with the claim that at least some pre-lexical processing can be carried out in parallel with decoding the task cue . Experiment 2 provided evidence that such processing is restricted to pre-lexical levels . These data suggest that a task set is a necessary preliminary to lexical processing when reading aloud. (shrink)

In this paper, we explore best practices for asynchronous discussions in graduate online bioethics education. We explain that online approaches have advantages and challenges in contrast to in-person discussions. Online challenges are lack of visual or auditory cues and technical access. Advantages include extended opportunities for specific focus, thoughtful reflection, and critical review. We found no significant review of related best practices in bioethics. Our more general literature review of graduate education and online approaches, plus experience in our own (...) bioethics graduate program, suggest provisional best practices that we detail. We reason that online graduate discussion provisionally should aim for a “community of inquiry” framework that incorporates cognitive, social, and teaching “presences,” as well as a learning presence. However, we also note unresolved concerns about whether the framework sufficiently addresses learning, is complete, and captures communicative functions. Drawing further on the literature, we also suggest best practices for instructor feedback to students about their discussion performance, including that remarks should be timely and specific. Summaries from two courses in our graduate bioethics program illustrate how we implement discussion strategies. Finally, we review assessment strategies and suggest that embedded formative assessments in discussion support and demonstrate deeper learning. Limitations in generalizability include that our review targets asynchronous and text-based online discussion, our program assessments do not necessarily detail best practices, student expectations and approaches could be rather particular to our program, and specific bioethics content in principle could alter best practices. (shrink)

While influential works since the 1970s have widely assumed that imitation is an innate skill in both human and non-human primate neonates, recent empirical studies and meta-analyses have challenged this view, indicating other forms of reward-based learning as relevant factors in the development of social behavior. The visual input translation into matching motor output that underlies imitation abilities instead seems to develop along with social interactions and sensorimotor experience during infancy and childhood. Recently, a new visual stream has been identified (...) in both human and non-human primate brains, updating the dual visual stream model. This third pathway is thought to be specialized for dynamics aspects of social perceptions such as eye-gaze, facial expression and crucially for audio-visual integration of speech. Here, we review empirical studies addressing an understudied but crucial aspect of speech and communication, namely the processing of visual orofacial cues and its integration with vocal auditory cues. Along this review, we offer new insights from our understanding of speech as the product of evolution and development of a rhythmic and multimodal organization of sensorimotor brain networks, supporting volitional motor control of the upper vocal tract and audio-visual voices-faces integration. (shrink)

Humour is a prevalent feature in any form of human interaction, regardless of language modality. This article explores in detail how humour is negotiated in conversations among deafblind Australians who are fluent users of tactile Australian Sign Language. Without access to the visual or auditory cues that are normally associated with humour, there is a risk that deafblind interactants will misconstrue humorous utterances as serious, or be unsure whether their conversation partner has got the joke. In this article, we (...) explore how humorous utterances unfold in tactile signed interactions. Drawing on Conversation Analytic principles, we outline the ad hoc and more conventionalised signals deafblind signers use to signal amusement. Looking at humour in these conversations contributes to a greater understanding of how humour is conveyed across language modalities and further support for humour’s centrality to interactional solidarity. (shrink)

Recent research into how musicians coordinate their expressive timing, phrasing, articulation, dynamics, and other stylistic characteristics during performances has highlighted the role of predictive processes, as musicians must anticipate how their partners will play in order to be together. Several studies have used information flow techniques such as Granger causality to show that upcoming movements of a musician can be predicted from immediate past movements of fellow musicians. Although musicians must move to play their instruments, a major goal of music (...) making is to create a joint interpretation through the sounds they produce. Yet, information flow techniques have not been applied previously to examine the role that fellow musicians' sound output plays in these predictive processes and whether this changes as they learn to play together. In the present experiment, we asked professional violinists to play along with recordings of two folk pieces, each eight times in succession, and compared the amplitude envelopes of their performances with those of the recordings using Granger causality to measure information flow and cross-correlation to measure similarity and synchronization. In line with our hypotheses, our measure of information flow was higher from the recordings to the performances than vice versa, and decreased as the violinists became more familiar with the recordings over trials. This decline in information flow is consistent with a gradual shift from relying on auditory cues to predict the recording to relying on an internally-based (learned) model built through repetition. There was also evidence that violinists became more synchronized with the recordings over trials. These results shed light on the planning and learning processes involved in the aligning of expressive intentions in group music performance and lay the groundwork for the application of Granger causality to investigate information flow through sound in more complex musical interactions. (shrink)

Multimodal integration is the formation of a coherent percept from different sensory inputs such as vision, audition, and somatosensation. Most research on multimodal integration in speech perception has focused on audio-visual integration. In recent years, audio-tactile integration has also been investigated, and it has been established that puffs of air applied to the skin and timed with listening tasks shift the perception of voicing by naive listeners. The current study has replicated and extended these findings by testing the effect of (...) air puffs on gradations of voice onset time along a continuum rather than the voiced and voiceless endpoints of the original work. Three continua were tested: bilabial (“pa/ba”), velar (“ka/ga”), and a vowel continuum (“head/hid”) used as a control. The presence of air puffs was found to significantly increase the likelihood of choosing voiceless responses for the two VOT continua but had no effect on choices for the vowel continuum. Analysis of response times revealed that the presence of air puffs lengthened responses for intermediate (ambiguous) stimuli and shortened them for endpoint (non-ambiguous) stimuli. The slowest response times were observed for the intermediate steps for all three continua, but for the bilabial continuum this effect interacted with the presence of air puffs: responses were slower in the presence of air puffs, and faster in their absence. This suggests that during integration auditory and aero-tactile inputs are weighted differently by the perceptual system, with the latter exerting greater influence in those cases where the auditory cues for voicing are ambiguous. (shrink)

Despite the power of VR in immersing viewers in an experience, it generally only targets viewers via visual and auditory cues. Human beings use more senses to gather information, so expectedly, the full potential of this medium is currently not yet tapped. This study contributes in answering two research questions: How can conventional VR ads be enriched by also addressing the forgotten sense of smell?; and Does doing so indeed instill more engaging experiences? A 2 × 3 between-subjects study (...) is conducted, whereby an existing branded VR commercial is augmented with “sound” and “scents.” The power of these sensory augmentations is evaluated by inspecting emotional, cognitive and conative dimensions of customer engagement. The results identify product-scent congruence as a deal-maker, albeit product-scent incongruence is not necessarily a deal-breaker. The article concludes with further research avenues and a translation into managerial implications. (shrink)