As Virtual Reality headsets become accessible, more and more artistic applications are developed, including immersive musical instruments. 3D interaction techniques designed in the 3D User Interfaces research community, such as navigation, selection and manipulation techniques, open numerous opportunities for musical control. For example, navigation techniques such as teleportation, free walking/flying and path-planning enable different ways of accessing musical scores, scenes of spatialised sound sources or even parameter spaces. Manipulation techniques provide novel gestures and metaphors, e.g. for drawing or sculpting sound entities. Finally, 3D selection techniques facilitate the interaction with complex visual structures which can represent hierarchical temporal structures, audio graphs, scores or parameter spaces. However, existing devices and techniques were developed mainly with a focus on efficiency, i.e. minimising error rate and task completion times. They were therefore not designed with the specifics of musical interaction in mind. In this paper, we review existing 3D interaction techniques and examine how they can be used for musical control, including the possibilities they open for instrument designers. We then propose a number of research directions to adapt and extend 3DUIs for musical expression

中文翻译：

---

用于音乐表达的 3D 交互技术

随着虚拟现实耳机的普及，越来越多的艺术应用被开发出来，包括沉浸式乐器。在 3D 用户界面研究社区中设计的 3D 交互技术，例如导航、选择和操作技术，为音乐控制开辟了许多机会。例如，诸如隐形传送、自由行走/飞行和路径规划之类的导航技术支持以不同方式访问乐谱、空间化声源场景甚至参数空间。操纵技术提供了新颖的手势和隐喻，例如用于绘制或雕刻声音实体。最后，3D 选择技术促进了与复杂视觉结构的交互，这些结构可以表示分层时间结构、音频图、分数或参数空间。然而，现有设备和技术的开发主要侧重于效率，即最小化错误率和任务完成时间。因此，它们的设计并没有考虑到音乐互动的细节。在本文中，我们回顾了现有的 3D 交互技术，并研究了如何将它们用于音乐控制，包括它们为乐器设计师带来的可能性。然后，我们提出了许多研究方向来适应和扩展 3DUI 以进行音乐表达 包括它们为仪器设计师打开的可能性。然后，我们提出了许多研究方向来适应和扩展 3DUI 以进行音乐表达 包括它们为仪器设计师打开的可能性。然后，我们提出了许多研究方向来适应和扩展 3DUI 以进行音乐表达