Sound symbolism refers to non‐arbitrary mappings between the sounds of words and their meanings and is often studied by pairing auditory pseudowords such as “maluma” and “takete” with rounded and pointed visual shapes, respectively. However, it is unclear what auditory properties of pseudowords contribute to their perception as rounded or pointed. Here, we compared perceptual ratings of the roundedness/pointedness of large sets of pseudowords and shapes to their acoustic and visual properties using a novel application of representational similarity analysis (RSA). Representational dissimilarity matrices (RDMs) of the auditory and visual ratings of roundedness/pointedness were significantly correlated crossmodally. The auditory perceptual RDM correlated significantly with RDMs of spectral tilt, the temporal fast Fourier transform (FFT), and the speech envelope. Conventional correlational analyses showed that ratings of pseudowords transitioned from rounded to pointed as vocal roughness (as measured by the harmonics‐to‐noise ratio, pulse number, fraction of unvoiced frames, mean autocorrelation, shimmer, and jitter) increased. The visual perceptual RDM correlated significantly with RDMs of global indices of visual shape (the simple matching coefficient, image silhouette, image outlines, and Jaccard distance). Crossmodally, the RDMs of the auditory spectral parameters correlated weakly but significantly with those of the global indices of visual shape. Our work establishes the utility of RSA for analysis of large stimulus sets and offers novel insights into the stimulus parameters underlying sound symbolism, showing that sound‐to‐shape mapping is driven by acoustic properties of pseudowords and suggesting audiovisual cross‐modal correspondence as a basis for language users' sensitivity to this type of sound symbolism.

中文翻译：

---

听觉伪词到视觉形状的声音符号映射的刺激参数。


声音象征主义是指单词的声音与其含义之间的非任意映射，通常通过将“maluma”和“takete”等听觉伪词分别与圆形和尖形视觉形状配对来研究。然而，目前还不清楚伪词的听觉特性是什么导致它们被感知为圆形或尖锐。在这里，我们使用表征相似性分析（RSA）的新颖应用，将大量伪词和形状的圆度/尖锐度的感知评级与其声学和视觉属性进行了比较。圆度/尖度的听觉和视觉评级的代表性相异矩阵（RDM）在跨模态上显着相关。听觉感知 RDM 与频谱倾斜、时间快速傅立叶变换 (FFT) 和语音包络的 RDM 显着相关。传统的相关分析表明，随着声音粗糙度（通过谐波噪声比、脉冲数、清音帧分数、平均自相关、闪烁和抖动来衡量）的增加，伪词的评级从圆形转变为尖锐。视觉感知 RDM 与视觉形状全局指数（简单匹配系数、图像轮廓、图像轮廓和杰卡德距离）的 RDM 显着相关。在跨模态上，听觉频谱参数的 RDM 与视觉形状全局指数的 RDM 相关性较弱，但显着相关。 我们的工作确立了 RSA 在分析大型刺激集方面的实用性，并对声音象征背后的刺激参数提供了新颖的见解，表明声音到形状的映射是由伪词的声学特性驱动的，并建议视听跨模态对应作为基础用于语言使用者对这种类型的声音象征的敏感性。