In this article we describe a user-driven adaptive method to control the sonic response of digital musical instruments using information extracted from the timbre of the human voice. The mapping between heterogeneous attributes of the input and output timbres is determined from data collected through machine-listening techniques and then processed by unsupervised machine-learning algorithms. This approach is based on a minimum-loss mapping that hides any synthesizer-specific parameters and that maps the vocal interaction directly to perceptual characteristics of the generated sound. The mapping adapts to the dynamics detected in the voice and maximizes the timbral space covered by the sound synthesizer. The strategies for mapping vocal control to perceptual timbral features and for automating the customization of vocal interfaces for different users and synthesizers, in general, are evaluated through a variety of qualitative and quantitative methods.

中文翻译：

---

通过无监督机器聆听和学习个性化声音合成的人声控制

在本文中，我们描述了一种用户驱动的自适应方法，该方法使用从人声音色中提取的信息来控制数字乐器的声音响应。输入和输出音色的异构属性之间的映射是根据通过机器监听技术收集的数据确定的，然后由无监督的机器学习算法处理。这种方法基于最小损失映射，该映射隐藏了任何特定于合成器的参数，并将人​​声交互直接映射到生成声音的感知特征。该映射适应在语音中检测到的动态并最大化声音合成器覆盖的音色空间。