Dance and music are two highly correlated artistic forms. Synthesizing dance motions has attracted much attention recently. Most previous works conduct music-to-dance synthesis via directly music to human skeleton keypoints mapping. Meanwhile, human choreographers design dance motions from music in a two-stage manner: they firstly devise multiple choreographic dance units (CAUs), each with a series of dance motions, and then arrange the CAU sequence according to the rhythm, melody and emotion of the music. Inspired by these, we systematically study such two-stage choreography approach and construct a dataset to incorporate such choreography knowledge. Based on the constructed dataset, we design a two-stage music-to-dance synthesis framework ChoreoNet to imitate human choreography procedure. Our framework firstly devises a CAU prediction model to learn the mapping relationship between music and CAU sequences. Afterwards, we devise a spatial-temporal inpainting model to convert the CAU sequence into continuous dance motions. Experimental results demonstrate that the proposed ChoreoNet outperforms baseline methods (0.622 in terms of CAU BLEU score and 1.59 in terms of user study score).

中文翻译：

---

ChoreoNet：通过编舞动作单元实现音乐到舞蹈的合成

舞蹈和音乐是两种高度相关的艺术形式。合成舞蹈动作近来备受关注。以前的大多数作品都是通过直接将音乐映射到人体骨骼关键点来进行音乐到舞蹈的合成。同时，人类编舞者以两阶段的方式从音乐中设计舞蹈动作：他们首先设计多个编舞舞蹈单元（CAU），每个舞蹈单元都有一系列舞蹈动作，然后根据节奏、旋律和情感安排 CAU 序列。音乐。受这些启发，我们系统地研究了这种两阶段编舞方法，并构建了一个数据集来整合这种编舞知识。基于构建的数据集，我们设计了一个两阶段的音乐到舞蹈合成框架 ChoreoNet 来模仿人类编舞过程。我们的框架首先设计了一个 CAU 预测模型来学习音乐和 CAU 序列之间的映射关系。之后，我们设计了一个时空修复模型，将 CAU 序列转换为连续的舞蹈动作。实验结果表明，所提出的 ChoreoNet 优于基线方法（CAU BLEU 得分为 0.622，用户研究得分为 1.59）。