Cross-modal correspondence is the tendency to systematically map stimulus features across sensory modalities. The current study explored cross-modal correspondence between speech sound and shape (Experiment 1), and whether such association can influence shape representation (Experiment 2). For the purpose of closely examining the role of the two factors - articulation and pitch - combined in speech acoustics, we generated two sets of 25 vowel stimuli - pitch-varying and pitch-constant sets. Both sets were generated by manipulating articulation - frontness and height of the tongue body's positions - but differed in terms of whether pitch varied among the sounds within the same set. In Experiment 1, participants made a forced choice between a round and a spiky shape to indicate the shape better associated with each sound. Results showed that shape choice was modulated according to both articulation and pitch, and we therefore concluded that both factors play significant roles in sound-shape correspondence. In Experiment 2, participants reported their subjective experience of shape accompanied by vowel sounds by adjusting an ambiguous shape in the response display. We found that sound-shape correspondence exerts an effect on shape representation by modulating audiovisual interaction, but only in the case of pitch-varying sounds. Therefore, pitch information within vowel acoustics plays the leading role in sound-shape correspondence influencing shape representation. Taken together, our results suggest the importance of teasing apart the roles of articulation and pitch for understanding sound-shape correspondence.

中文翻译：

---

语音和视觉形状之间的跨模态对应影响形状的感知表示：清晰度和音高的作用。

跨模式对应是系统地映射跨感官模式的刺激特征的趋势。当前的研究探索了语音和形状之间的跨模态对应（实验 1），以及这种关联是否会影响形状表示（实验 2）。为了仔细检查两个因素 - 清晰度和音高 - 在语音声学中的作用，我们生成了两组 25 个元音刺激 - 音高变化和音高常数集。两组都是通过操纵发音（舌体位置的前部和高度）生成的，但在同一组内的声音之间的音高是否不同方面有所不同。在实验 1 中，参与者在圆形和尖刺形状之间做出强制选择，以指示与每种声音更相关的形状。结果表明，形状选择是根据发音和音高进行调制的，因此我们得出结论，这两个因素在声音-形状对应中都起着重要作用。在实验 2 中，参与者通过调整响应显示中的模糊形状来报告他们对伴随元音声音的形状的主观体验。我们发现声音-形状对应通过调节视听交互对形状表示产生影响，但仅限于音高变化的声音。因此，元音声学中的音高信息在影响形状表示的音形对应中起主导作用。综上所述，我们的结果表明，为了理解音形对应关系，区分清晰度和音高的作用是很重要的。