Stimulus Parameters Underlying Sound‐Symbolic Mapping of Auditory Pseudowords to Visual Shapes

Cognitive Science 44 (9):e12883 (2020)
  Copy   BIBTEX

Abstract

Sound symbolism refers to non‐arbitrary mappings between the sounds of words and their meanings and is often studied by pairing auditory pseudowords such as “maluma” and “takete” with rounded and pointed visual shapes, respectively. However, it is unclear what auditory properties of pseudowords contribute to their perception as rounded or pointed. Here, we compared perceptual ratings of the roundedness/pointedness of large sets of pseudowords and shapes to their acoustic and visual properties using a novel application of representational similarity analysis (RSA). Representational dissimilarity matrices (RDMs) of the auditory and visual ratings of roundedness/pointedness were significantly correlated crossmodally. The auditory perceptual RDM correlated significantly with RDMs of spectral tilt, the temporal fast Fourier transform (FFT), and the speech envelope. Conventional correlational analyses showed that ratings of pseudowords transitioned from rounded to pointed as vocal roughness (as measured by the harmonics‐to‐noise ratio, pulse number, fraction of unvoiced frames, mean autocorrelation, shimmer, and jitter) increased. The visual perceptual RDM correlated significantly with RDMs of global indices of visual shape (the simple matching coefficient, image silhouette, image outlines, and Jaccard distance). Crossmodally, the RDMs of the auditory spectral parameters correlated weakly but significantly with those of the global indices of visual shape. Our work establishes the utility of RSA for analysis of large stimulus sets and offers novel insights into the stimulus parameters underlying sound symbolism, showing that sound‐to‐shape mapping is driven by acoustic properties of pseudowords and suggesting audiovisual cross‐modal correspondence as a basis for language users' sensitivity to this type of sound symbolism.

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 91,139

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Cross-modal iconicity.Felix Ahlner & Jordan Zlatev - 2010 - Sign Systems Studies 38 (1-4):298-346.
Visual and auditory coding in a memory matching task.Larry E. Wood - 1974 - Journal of Experimental Psychology 102 (1):106.

Analytics

Added to PP
2020-09-11

Downloads
11 (#1,065,379)

6 months
7 (#328,545)

Historical graph of downloads
How can I increase my downloads?