According to the hierarchical model of sensory information processing, sensory inputs are transmitted to cortical areas, which are crucial for complex auditory and speech processing, only after being processed in subcortical areas (Hickok and Poeppel, 2007; Rauschecker and Scott, 2009). However, studies using electroencephalography (EEG) indicate that distinguishing simultaneous auditory inputs involves a widely distributed neural network, including the medial temporal lobe, which is essential for declarative memory, and posterior association cortices (Alain et al., 2001; Squire et al., (...) 2004). More recent studies have even demonstrated plasticity of auditory signals as low as the brainstem (Suga, 2008). Collectively, studies suggest that the functional architecture of perceptual processing involves primarily top-down modulation (Suga et al., 2002; Gilbert and Li, 2013; Chandrasekaran et al., 2014). Top-down influences exerted throughout the auditory systems (Lotto and Holt, 2011) include: memory (Goldinger, 1998)1, attention (Choi et al., 2014), which has been found to modulate auditory encoding in the cochlea, a subcortical area (Maison et al., 2001), (prior) knowledge of syntax or words (Ganong, 1980; Warren, 1984)2, and experience-based expectations pertaining to the speaker's accent (Deutsch, 1996; Deutsch et al., 2004; Irino and Patterson, 2006), gender (Johnson et al., 1999), and vocal folds or tract (Irino and Patterson, 2002; Patterson and Johnsrude, 2008) . (shrink)

Songbirds possess mirror neurons (MNs) activating during the perception and execution of specific features of songs. These neurons are located in high vocal center (HVC), a premotor nucleus implicated in song perception, production and learning, making worth to inquire their properties and functions in vocal recognition and imitative learning. By integrating a body of brain and behavioral data, we discuss neurophysiology, anatomical, computational properties and possible functions of songbird MNs. -/- We state that the neurophysiological properties of songbird MNs depends (...) on sensorimotor regions that are outside the auditory neural system. Interestingly, songbirds MNs can be the result of the specific type of song representation possessed by some songbird species. At the functional level, we discuss whether songbird MNs are involved in others' song recognition, by dissecting the function of recognition in various different but possible overlapping processes: action-oriented perception, discriminative-oriented perception and identification of the signaler. We conclude that songbird MNs may be involved in recognizing other singer's vocalizations, while their role in imitative learning still require to solve how auditory feedback are used to correct own vocal performance to match the tutor song. Finally, we compare songbird and human mirror responses, hypothesizing a case of convergent evolution, and proposing new experimental directions. (shrink)

A neuropsychological perspective on auditory verbal hallucinations links key phenomenological features of the experience, such as voice location and identity, to functionally separable pathways in normal human audition. Although this auditory processing stream framework has proven valuable for integrating research on phenomenology with cognitive and neural accounts of hallucinatory experiences, it has not yet been applied to other symptoms presumed to be closely related to AVH – such as thought insertion. In this paper, I propose that an APS (...) framework offers a useful way of thinking about the experience of TI as well as AVH, providing a common conceptual framework for both. I argue that previous self-monitoring theories struggle to account for both the differences and similarities in the characteristic features of AVH and TI, which can be readily accommodated within an APS framework. Furthermore, the APS framework can be integrated with predictive processing accounts of psychotic symptoms; makes predictions about potential sites of prediction error signals; and may offer a template for understanding a range of other symptoms beyond AVH and TI. (shrink)

Mismatch negativity is the electroencephalographic waveform obtained by subtracting event-related potential responses evoked by unexpected deviant stimuli from responses evoked by expected standard stimuli. While the MMN is thought to reflect an unexpected change in an ongoing, predictable stimulus, it is unknown whether MMN responses evoked by changes in different stimulus features have different magnitudes, latencies, and topographies. The present study aimed to investigate whether MMN responses differ depending on whether sudden stimulus change occur in pitch, duration, location or vowel (...) identity, respectively. To calculate ERPs to standard and deviant stimuli, EEG signals were recorded in normal-hearing participants who listened to roving oddball sequences of artificial syllables. In the roving paradigm, any given stimulus is repeated several times to form a standard, and then suddenly replaced with a deviant stimulus which differs from the standard. Here, deviants differed from preceding standards along one of four features. The feature levels were individually chosen to match behavioral discrimination performance. We identified neural activity evoked by unexpected violations along all four acoustic dimensions. Evoked responses to deviant stimuli increased in amplitude relative to the responses to standard stimuli. A univariate analysis yielded no significant differences between MMN responses following violations of different features. However, in a multivariate analysis, acoustic features could be decoded from the topography of mismatch responses, although at later latencies than those typical for MMN. These results support the notion that deviant feature detection may be subserved by a different process than general mismatch detection. (shrink)

We can easily discriminate self-produced from externally generated sensory signals. Recent studies suggest that the prediction of the sensory consequences of one’s own actions made by forward model can be used to attenuate the sensory effects of self-produced movements, thereby enabling a differentiation of the self-produced sensation from the externally generated one. The present study showed that attenuation of sensation occurred both when participants themselves performed a goal-directed action and when they observed experimenter performing the same action, although they (...) clearly reported that the tones were produced by other during action observation and by themselves during their own action. These results suggest that sensory prediction of action modulates ongoing auditory processing irrespective of who produces the sounds and that the explicit judgment of agency does not necessarily rely on the same mechanisms on which implicit perceptual measures such as sensory attenuation rely. (shrink)

In this report, the phenomenology of two blind users of a sensory substitution device – “The vOICe” – that converts visual images to auditory signals is described. The users both report detailed visual phenomenology that developed within months of immersive use and has continued to evolve over a period of years. This visual phenomenology, although triggered through use of The vOICe, is likely to depend not only on online visualization of the auditory signal but also on the (...) users’ previous (albeit distant) experience of veridical vision (e.g. knowledge of shapes and visual perspective). Once established, the sensory substitution mapping between the auditory and visual domains is not confined to when the device is worn and, thus, may constitute an example of acquired synaesthesia. (shrink)

Three experiments investigated the nature of visuo-auditory crossmodal cueing in a triadic setting: participants had to detect an auditory signal while observing another agent’s head facing one of the two laterally positioned auditory sources. Experiment 1 showed that when the agent’s eyes were open, sounds originating on the side of the agent’s gaze were detected faster than sounds originating on the side of the agent’s visible ear; when the agent’s eyes were closed this pat-tern of responses was (...) reversed. Two additional experiments showed that the results were sensitive to whether participants could infer a hearing function on the part of the agent. When no ear was depicted on the agent, only a gaze-side advantage was observed , but when the agent’s ear was covered , an ear side advantage was observed only when hearing could still be inferred but not when hearing was inferred to be diminished . The findings are discussed in the context of inferential and simulation processes and joint attention mechanisms. (shrink)

Excessive dog barking is among the leading sources of noise pollution world-wide; however, the reasons for the annoyance of barking to people remained uninvestigated. Our questions were: is the annoyance rating affected by the acoustic parameters of barks; does the attributed inner state of the dog and the nuisance caused by its barks correlate; does the gender and country of origin affect the subjects’ sensitivity to barking. Participants from Hungary (N = 100) and Brazil (N = 60) were tested with (...) sets of 27 artificial bark sequences. Subjects rated each bark according to the inner state of the dog and the annoyance caused by the particular bark. Subjects from both countries found high-pitched barks the most annoying: however, harsh, fast-pulsing, low-pitched barks were also unpleasant. Men found high-pitched barks more annoying than the women did. Annoyance ratings showed positive correlation with assumed negative inner states of the dog, positive emotional ratings showed negative correlation with the annoyance level. This is the first indication that acoustic features that were selected for effective vocal signalling may be annoying for human listeners. Among the explanations for this effect the role of affective communication and similar bioacoustics of particular animal vocalizations and baby cries are discussed. (shrink)

Arbitrary communication systems can emerge from iconic beginnings through processes of conventionalisation via interaction. Here, we explore whether this process of conventionalisation occurs with continuous, auditory signals. We conducted an artificial signalling experiment. Participants either created signals for themselves, or for a partner in a communication game. We found no evidence that the speech-like signals in our experiment became less iconic or simpler through interaction. We hypothesise that the reason for our results is that when it (...) is difficult to be iconic initially because of the constraints of the modality, then iconicity needs to emerge to enable grounding before conventionalisation can occur. Further, pressures for discrimination, caused by the expanding meaning space in our study, may cause more complexity to emerge, again as a result of the restrictive signalling modality. Our findings have possible implications for the processes of conventionalisation possible in signed and spoken languages, as the spoken modality is more restrictive than the manual modality. (shrink)

Neuroethological investigations of mammalian and avian auditory systems have documented species-specific specializations for processing complex acoustic signals that could, if viewed in abstract terms, have an intriguing and striking relevance for human speech sound categorization and representation. Each species forms biologically relevant categories based on combinatorial analysis of information-bearing parameters within the complex input signal. This target article uses known neural models from the mustached bat and barn owl to develop, by analogy, a conceptualization of human processing of (...) consonant plus vowel sequences that offers a partial solution to the noninvariance dilemma the locus equations orderly output constraint”) is hypothesized based on the notion of an evolutionarily conserved auditory-processing strategy. High correlation and linearity between critical parameters in the speech signal that help to cue place of articulation categories might have evolved to satisfy a preadaptation by mammalian auditory systems for representing tightly correlated, linearly related components of acoustic signals. (shrink)

In the social world, multiple sensory channels are used concurrently to facilitate communication. Among human and nonhuman pri- mates, faces and voices are the primary means of transmitting social signals (Adolphs, 2003; Ghazanfar and Santos, 2004). Primates recognize the correspondence between species-specific facial and vocal expressions (Massaro, 1998; Ghazanfar and Logothetis, 2003; Izumi and Kojima, 2004), and these visual and auditory channels can be integrated into unified percepts to enhance detection and discrimination. Where and how such communication (...) class='Hi'>signals are integrated at the neural level are poorly understood. In particular, it is unclear what role “unimodal” sensory areas, such as the auditory cortex, may play. We recorded local field potential activity, the signal that best correlates with human imaging and event-related potential signals, in both the core and lateral belt regions of the auditory cortex in awake behaving rhesus monkeys while they viewed vocalizing conspecifics. We demonstrate unequivocally that the primate auditory cortex integrates facial and vocal signals through enhancement and suppression of field potentials in both the core and lateral belt regions. The majority of these multisensory responses were specific to face/voice integration, and the lateral belt region shows a greater frequency of multisensory integration than the core region. These multisensory processes in the auditory cortex likely occur via recip- rocal interactions with the superior temporal sulcus. (shrink)

When you hear a person speaking in a familiar language you perceive the speech sounds uttered and the voice that produces them. How are speech sounds and voice related in a typical auditory experience of hearing speech in a particular voice? And how to conceive of the objects of such experiences? I propose a conception of auditory objects of speech perception as temporally structured mereologically complex individuals. A common experience is that speech sounds and the voice that produces (...) them appear united. I argue that the metaphysical underpinnings of the experienced unity of speech sounds and voices can be explained in terms of the mereological view on sounds and their sources. I also propose a psychological explanation of how we form and individuate the auditory objects of experiences of listening to speech in a particular voice. Voice characteristics enable determining the identity of auditory objects of speech sound perception by making some features of the speech signal stable and predictable. (shrink)

Auditory warnings have been shown to interfere with verbal working memory. However, the impact of different types of auditory warnings on working memory tasks must be further researched. This study investigated how different kinds of auditory warnings interfered with verbal and spatial working memory. Experiment 1 tested the potential interference of auditory warnings with verbal working memory. Experiment 2 tested the potential interference of auditory warnings with spatial working memory. Both experiments used a 3 × (...) 3 mixed design: auditory warning type was between groups, and task condition was within groups. In Experiment 1, earcons and spearcons but not auditory icons worsened the performance on the verbal serial recall task in the identify-warning condition, compared with that in the no-warning or ignore-warning conditions. In Experiment 2, only identifying earcons worsened the performance on the location recall task compared with performance without auditory warnings or when auditory warnings were ignored. Results are discussed from the perspective of working memory resource interference, and their practical application in the selection and design of auditory warning signals is involved. (shrink)

This investigation examined age-related differences in auditory-visual integration as reflected on perceptual judgments of temporally misaligned AV English sentences spoken by native English and native Spanish talkers. In the detection task, it was expected that slowed auditory temporal processing of older participants, relative to younger participants, would be manifest as a shift in the range over which participants would judge asynchronous stimuli as synchronous. The older participants were also expected to exhibit greater declines in speech recognition for asynchronous (...) AV stimuli than younger participants. Talker accent was hypothesized to influence listener performance, with older listeners exhibiting a greater narrowing of the AV simultaneity window and much poorer recognition of asynchronous AV foreign-accented speech compared to younger listeners. Participant groups included younger and older participants with normal hearing and older participants with hearing loss. Stimuli were video recordings of sentences produced by native English and native Spanish talkers. The video recordings were altered in 50 ms steps by delaying either the audio or video onset. Participants performed a detection task in which they judged whether the sentences were synchronous or asynchronous, and performed a recognition task for multiple synchronous and asynchronous conditions. Both the detection and recognition tasks were conducted at the individualized signal-to-noise ratio corresponding to approximately 70% correct speech recognition performance for synchronous AV sentences. Older listeners with and without hearing loss generally showed wider AV simultaneity windows than younger listeners, possibly reflecting slowed auditory temporal processing in auditory lead conditions and reduced sensitivity to asynchrony in auditory lag conditions. However, older and younger listeners were affected similarly by misalignment of auditory and visual signal onsets on the speech recognition task. This suggests that older listeners are negatively impacted by temporal misalignments for speech recognition, even when they do not notice that the stimuli are asynchronous. Overall, the findings show that when listener performance is equated for simultaneous AV speech signals, age effects are apparent in detection judgments but not in recognition of asynchronous speech. (shrink)

Dealing with phonological variations is important for speech processing. This article addresses whether phonological variations introduced by assimilatory processes are compensated for at the pre-lexical or lexical level, and whether the nature of variation and the phonological context influence this process. To this end, Swedish nasal regressive place assimilation was investigated using the mismatch negativity component. In nasal regressive assimilation, the coronal nasal assimilates to the place of articulation of a following segment, most clearly with a velar or labial place (...) of articulation, as in utan mej “without me” > [ʉːtam mɛjː]. In a passive auditory oddball paradigm, 15 Swedish speakers were presented with Swedish phrases with attested and unattested phonological variations and contexts for nasal assimilation. Attested variations – a coronal-to-labial change as in utan “without” > [ʉːtam] – were contrasted with unattested variations – a labial-to-coronal change as in utom “except” > ∗[ʉːtɔn] – in appropriate and inappropriate contexts created by mej “me” [mɛjː] and dej “you” [dɛjː]. Given that the MMN amplitude depends on the degree of variation between two stimuli, the MMN responses were expected to indicate to what extent the distance between variants was tolerated by the perceptual system. Since the MMN response reflects not only low-level acoustic processing but also higher-level linguistic processes, the results were predicted to indicate whether listeners process assimilation at the pre-lexical and lexical levels. The results indicated no significant interactions across variations, suggesting that variations in phonological forms do not incur any cost in lexical retrieval; hence such variation is compensated for at the lexical level. However, since the MMN response reached significance only for a labial-to-coronal change in a labial context and for a coronal-to-labial change in a coronal context, the compensation might have been influenced by the nature of variation and the phonological context. It is therefore concluded that while assimilation is compensated for at the lexical level, there is also some influence from pre-lexical processing. The present results reveal not only signal-based perception of phonological units, but also higher-level lexical processing, and are thus able to reconcile the bottom-up and top-down models of speech processing. (shrink)

People are ingesting various information from different sense organs all the time to complete different cognitive tasks. The brain integrates and regulates this information. The two significant sensory channels for receiving external information are sight and hearing that have received extensive attention. This paper mainly studies the effect of music and visual-auditory stimulation on electroencephalogram of happy emotion recognition based on a complex system. In the experiment, the presentation was used to prepare the experimental stimulation program, and the cognitive (...) neuroscience experimental paradigm of EEG evoked by happy emotion pictures was established. Using 93 videos as natural stimuli, fMRI data were collected. Finally, the collected EEG signals were removed with the eye artifact and baseline drift, and the t-test was used to analyze the significant differences of different lead EEG data. Experimental data shows that, by adjusting the parameters of the convolutional neural network, the highest accuracy of the two-classification algorithm can reach 98.8%, and the average accuracy can reach 83.45%. The results show that the brain source under the combined visual and auditory stimulus is not a simple superposition of the brain source of the single visual and auditory stimulation, but a new interactive source is generated. (shrink)

Listeners can attend to and track instruments or singing voices in complex musical mixtures, even though the acoustical energy of sounds from individual instruments may overlap in time and frequency. In popular music, lead vocals are often accompanied by sound mixtures from a variety of instruments, such as drums, bass, keyboards, and guitars. However, little is known about how the perceptual organization of such musical scenes is affected by selective attention, and which acoustic features play the most important role. To (...) investigate these questions, we explored the role of auditory attention in a realistic musical scenario. We conducted three online experiments in which participants detected single cued instruments or voices in multi-track musical mixtures. Stimuli consisted of 2-s multi-track excerpts of popular music. In one condition, the target cue preceded the mixture, allowing listeners to selectively attend to the target. In another condition, the target was presented after the mixture, requiring a more “global” mode of listening. Performance differences between these two conditions were interpreted as effects of selective attention. In Experiment 1, results showed that detection performance was generally dependent on the target’s instrument category, but listeners were more accurate when the target was presented prior to the mixture rather than the opposite. Lead vocals appeared to be nearly unaffected by this change in presentation order and achieved the highest accuracy compared with the other instruments, which suggested a particular salience of vocal signals in musical mixtures. In Experiment 2, filtering was used to avoid potential spectral masking of target sounds. Although detection accuracy increased for all instruments, a similar pattern of results was observed regarding the instrument-specific differences between presentation orders. In Experiment 3, adjusting the sound level differences between the targets reduced the effect of presentation order, but did not affect the differences between instruments. While both acoustic manipulations facilitated the detection of targets, vocal signals remained particularly salient, which suggest that the manipulated features did not contribute to vocal salience. These findings demonstrate that lead vocals serve as robust attractor points of auditory attention regardless of the manipulation of low-level acoustical cues. (shrink)

Our research aimed to investigate the effectiveness of auditory, visual, and audiovisual warning signals for capturing the attention of the pilot, and how stimulus onset asynchronies in audiovisual stimuli affect pilots perceiving the bimodal warning signals under different perceptual load conditions. In experiment 1 of the low perceptual load condition, participants discriminated the location of visual targets preceded by five different types of warning signals. In experiment 2 of high perceptual load, participants completed the location task (...) identical to a low load condition and a digit detection task in a rapid serial visual presentation stream. The main effect of warning signals in two experiments showed that visual and auditory cues presented simultaneously could effectively and efficiently arouse the attention of the pilots in high and low load conditions. Specifically, auditory, AV, and visual preceding auditory stimulus by 100 ms increased the spatial orientation to a valid position in low load conditions. With the increase in visual perceptual load, auditory preceding the visual stimulus by 100 ms and A warning signals had stronger spatial orientation. The results are expected to theoretically support the optimization design of the cockpit display interface, contributing to immediate flight crew awareness. (shrink)

Recent computational models of perception conceptualize auditory oddball responses as signatures of a learning process, in line with the influential view of the mismatch negativity as a prediction error signal. Novel MMN experimental paradigms have put an emphasis on neurophysiological effects of manipulating regularity and predictability in sound sequences. This raises the question of the contextual adaptation of the learning process itself, which on the computational side speaks to the mechanisms of gain-modulated prediction error. In this study using electrocorticographic (...) signals, we manipulated the predictability of oddball sound sequences with two objectives: Uncovering the computational process underlying trial-by-trial variations of the cortical responses. The fluctuations between trials, generally ignored by approaches based on averaged evoked responses, should reflect the learning involved. We used a general linear model and Bayesian Model Reduction to assess the respective contributions of experimental manipulations and learning mechanisms under probabilistic assumptions. To validate and expand on previous findings regarding the effect of changes in predictability using simultaneous EEG-MEG recordings. Our trial-by-trial analysis revealed only a few stimulus-responsive sensors but the measured effects appear to be consistent over subjects in both time and space. In time, they occur at the typical latency of the MMN. In space, we found a dissociation between time-independent effects in more anterior temporal locations and time-dependent effects in more posterior locations. However, we could not observe any clear and reliable effect of our manipulation of predictability modulation onto the above learning process. Overall, these findings clearly demonstrate the potential of trial-to-trial modeling to unravel perceptual learning processes and their neurophysiological counterparts. (shrink)

The massive acoustic redundancy of minimally contrasting speech sounds, coupled with the auditory integration of psychoacoustically similar acoustic properties produces a highly invariant percept, which cannot be improved by top-down feedback from the lexicon. Contextual effects are also bottom-up but not all entirely auditory and may thus differ in whether they affect sensitivity or only response bias.