Modelling audiovisual integration of affect from videos and music

Cognition and Emotion 32 (3):516-529 (2017)
  Copy   BIBTEX


Two experiments examined how affective values from visual and auditory modalities are integrated. Experiment 1 paired music and videos drawn from three levels of valence while holding arousal constant. Experiment 2 included a parallel combination of three levels of arousal while holding valence constant. In each experiment, participants rated their affective states after unimodal and multimodal presentations. Experiment 1 revealed a congruency effect in which stimulus combinations of the same extreme valence resulted in more extreme state ratings than component stimuli presented in isolation. An interaction between music and video valence reflected the greater influence of negative affect. Video valence was found to have a significantly greater effect on combined ratings than music valence. The pattern of data was explained by a five parameter differential weight averaging model that attributed greater weight to the visual modality and increased weight with decreasing values of valence. Experiment 2 revealed a congruency effect only for high arousal combinations and no interaction effects. This pattern was explained by a three parameter constant weight averaging model with greater weight for the auditory modality and a very low arousal value for the initial state. These results demonstrate key differences in audiovisual integration between valence and arousal.



    Upload a copy of this work     Papers currently archived: 77,697

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Teaching Philosophy Using Music Videos.Charlene Tan - 2008 - Thinking: The Journal of Philosophy for Children 19 (1):44-48.
Implicit learning and acquisition of music.Martin Rohrmeier & Patrick Rebuschat - 2012 - Topics in Cognitive Science 4 (4):525-553.
On the Ancient Idea that Music Shapes Character.James Harold - 2016 - Dao: A Journal of Comparative Philosophy 15 (3):341-354.


Added to PP

11 (#853,077)

6 months
1 (#481,005)

Historical graph of downloads
How can I increase my downloads?