It is now widely accepted that the brunt of animal communication is conducted via several modalities, e.g. acoustic and visual, either simultaneously or sequentially. This is a laudable multimodal turn relative to traditional accounts of temporal aspects of animal communication which have focused on a single modality at a time. However, the fields that are currently contributing to the study of multimodal communication are highly varied, and still largely disconnected given their sole focus on a particular level of description or their particular concern with human or non-human animals. Here, we provide an integrative overview of converging findings that show how multimodal processes occurring at neural, bodily, as well as social interactional levels each contribute uniquely to the complex rhythms that characterize communication in human and non-human animals. Though we address findings for each of these levels independently, we conclude that the most important challenge in this field is to identify how processes at these different levels connect.

This article is part of the theme issue ‘Synchrony and rhythm interaction: from the brain to behavioural ecology’.

现在人们普遍认为，动物交流的冲击是通过多种方式进行的，例如同时或顺序的声音和视觉。相对于动物交流的时间方面的传统描述，这是一个值得称赞的多模态转变，后者一次只关注一种模态。然而，目前对多模式通信研究做出贡献的领域是多种多样的，并且由于它们只关注特定级别的描述或他们对人类或非人类动物的特别关注，因此在很大程度上仍然是脱节的。在这里，我们提供了对融合发现的综合概述，这些发现展示了多模态过程如何在神经、身体、以及社会互动水平，每一个都对人类和非人类动物交流的复杂节奏做出了独特的贡献。尽管我们独立处理每个级别的发现，但我们得出结论，该领域最重要的挑战是确定这些不同级别的流程如何连接。

这篇文章是主题问题“同步和节奏交互：从大脑到行为生态学”的一部分。