We memorize our daily life experiences, which are often multisensory in nature, by segmenting them into distinct event models, in accordance with perceived contextual or situational changes. However, very little is known about how multisensory boundaries affect segmentation, as most studies have focused on unisensory (visual or audio) segmentation. In three experiments, we investigated the effect of multisensory boundaries on segmentation in memory and perception. In Experiment 1, participants encoded lists of pictures while audio and visual contexts changed synchronously or asynchronously. After each list, we tested recognition and temporal associative memory for pictures that were encoded in the same audio-visual context or that crossed a synchronous or an asynchronous multisensory change. We found no effect of multisensory synchrony for recognition memory: synchronous and asynchronous changes similarly impaired recognition for pictures encoded at those changes, compared to pictures encoded further away from those changes. Multisensory synchrony did affect temporal associative memory, which was worse for pictures encoded at synchronous than at asynchronous changes. Follow up experiments showed that this effect was not due to the higher dimensionality of multisensory over unisensory contexts (Experiment 2), nor that it was due to the temporal unpredictability of contextual changes inherent to Experiment 1 (Experiment 3). We argue that participants formed situational expectations through multisensory synchronicity, such that synchronous multisensory changes deviated more strongly from those expectations than asynchronous changes. We discuss our findings in light of supportive and conflicting findings of uni- and multi-sensory segmentation.

我们根据感知到的上下文或情境变化，将日常生活经历分成不同的事件模型，从而记住我们的日常生活经历，这些经历本质上通常是多感官的。然而，关于多感官边界如何影响分割知之甚少，因为大多数研究都集中在单感官（视觉或音频）分割上。在三个实验中，我们研究了多感官边界对记忆和感知分割的影响。在实验 1 中，参与者对图片列表进行编码，同时音频和视觉上下文同步或异步变化。在每个列表之后，我们测试了在相同视听环境中编码的图片或跨越同步或异步多感官变化的图片的识别和时间联想记忆。我们发现多感官同步对识别记忆没有影响：与远离这些变化编码的图片相比，同步和异步变化同样会损害在这些变化时编码的图片的识别。多感官同步确实影响了时间联想记忆，这对于同步编码的图片比异步更改更糟糕。后续实验表明，这种效果不是由于多感官的维度高于单感官上下文（实验 2），也不是由于实验 1 固有的上下文变化的时间不可预测性（实验 3）。我们认为参与者通过多感官同步性形成情境期望，因此同步多感官变化比异步变化更强烈地偏离这些期望。