We present a controllable neural audio synthesizer based on Gaussian Mixture Variational Autoencoders (GM-VAE), which can generate realistic piano performances in the audio domain that closely follows temporal conditions of two essential style features for piano performances: articulation and dynamics. We demonstrate how the model is able to apply fine-grained style morphing over the course of synthesizing the audio. This is based on conditions which are latent variables that can be sampled from the prior or inferred from other pieces. One of the envisioned use cases is to inspire creative and brand new interpretations for existing pieces of piano music.

中文翻译：

---

富有表现力的钢琴演奏的可控音频合成的生成建模

我们提出了一种基于高斯混合变分自动编码器 (GM-VAE) 的可控神经音频合成器，它可以在音频域中生成逼真的钢琴演奏，紧跟钢琴演奏的两个基本风格特征的时间条件：清晰度和动态。我们演示了模型如何能够在合成音频的过程中应用细粒度的样式变形。这是基于潜在变量的条件，可以从先验中采样或从其他部分推断出来。设想的用例之一是激发对现有钢琴音乐的创造性和全新的诠释。