In this paper, we present a transformer-based learning framework for 3D dance generation conditioned on music. We carefully design our network architecture and empirically study the keys for obtaining qualitatively pleasing results. The critical components include a deep cross-modal transformer, which well learns the correlation between the music and dance motion; and the full-attention with future-N supervision mechanism which is essential in producing long-range non-freezing motion. In addition, we propose a new dataset of paired 3D motion and music called AIST++, which we reconstruct from the AIST multi-view dance videos. This dataset contains 1.1M frames of 3D dance motion in 1408 sequences, covering 10 genres of dance choreographies and accompanied with multi-view camera parameters. To our knowledge it is the largest dataset of this kind. Rich experiments on AIST++ demonstrate our method produces much better results than the state-of-the-art methods both qualitatively and quantitatively.

中文翻译：

---

学习与AIST ++一起跳舞：音乐调节型3D舞蹈生成

在本文中，我们提出了一种基于转换器的基于音乐的3D舞蹈生成学习框架。我们精心设计我们的网络架构，并根据经验研究获得定性令人愉悦的结果的关键。关键组件包括一个深层的交叉模态变压器，该变压器可以很好地了解音乐与舞蹈动作之间的相关性。并充分注意Future-N监督机制，这对于产生远程非冻结运动至关重要。此外，我们提出了一个称为AIST ++的3D运动和音乐配对的新数据集，该数据集是从AIST多视图舞蹈视频中重建的。该数据集包含1408个序列的110万帧3D舞蹈动作，涵盖10种舞蹈编排，并带有多视点摄影机参数。据我们所知，它是此类最大的数据集。