Many behaviors that are critical for survival and reproduction are expressed over extended time periods. The ability to inexpensively record and store large volumes of video data creates new opportunities to understand the biological basis of these behaviors and simultaneously creates a need for tools that can automatically quantify behaviors from large video datasets. Here, we demonstrate that 3D Residual Networks can be used to classify an array of complex behaviors in Lake Malawi cichlid fishes. We first apply pixel-based hidden Markov modeling combined with density-based spatiotemporal clustering to identify sand disturbance events. After this, a 3D ResNet, trained on 11,000 manually annotated video clips, accurately (>76%) classifies the sand disturbance events into 10 fish behavior categories, distinguishing between spitting, scooping, fin swipes, and spawning. Furthermore, animal intent can be determined from these clips, as spits and scoops performed during bower construction are classified independently from those during feeding.

许多对生存和繁殖至关重要的行为是在很长一段时间内表现出来的。廉价地记录和存储大量视频数据的能力为理解这些行为的生物学基础创造了新的机会，同时也产生了对能够自动量化大型视频数据集中的行为的工具的需求。在这里，我们演示了 3D 残差网络可用于对马拉维湖慈鲷鱼类的一系列复杂行为进行分类。我们首先应用基于像素的隐马尔可夫模型结合基于密度的时空聚类来识别沙扰事件。此后，3D ResNet 在 11,000 个手动注释的视频剪辑上进行训练，准确地 (>76%) 将沙扰事件分为 10 个鱼类行为类别，区分吐痰、舀鱼、挥鳍和产卵。此外，可以从这些剪辑中确定动物的意图，因为在凉亭建造期间进行的吐痰和舀勺的分类与喂食期间的吐痰和舀勺是独立的。