Touch-produced sounds in tool-surface interactions convey rich information about textured surface properties and provide direct feedback about how users interact with the surface. This article presents a statistical learning-based approach for modeling and rendering touch-produced sounds in real time. We apply a data-driven modeling method, which recreates highly realistic sounds using audio signals recorded from unconstrained tool-surface interactions. The recorded sound is segmented, and each segment is labeled with the average velocity during that time. We model each segment with wavelet tree models using a moving window approach. Each window is analyzed by fast wavelet transform and is then broken down into a tree structure. During rendering, we use the user's current velocity to select tree models and synthesize new sounds by breadth-first search and inverse wavelet transform. We conducted a user study to evaluate the realism of our virtual sounds and their effect on human's perception of the texture dimensions in the presence of simultaneous real haptic cues. The results showed that in the presence of haptic cues, the virtual sound can more completely capture the texture's roughness and hardness than haptic cues alone. However, the perception on slipperiness depended mainly on touch.

中文翻译：

---

迈向多感官感知：建模和渲染工具表面相互作用的声音

工具表面交互中的触摸产生的声音传达了有关纹理表面属性的丰富信息，并提供有关用户如何与表面交互的直接反馈。本文介绍了一种基于统计学习的方法，用于实时建模和渲染触摸产生的声音。我们应用了一种数据驱动的建模方法，该方法使用从不受约束的工具表面交互中记录的音频信号重新创建高度逼真的声音。录制的声音被分段，每个分段都标有该时间段的平均速度。我们使用移动窗口方法用小波树模型对每个段进行建模。每个窗口都通过快速小波变换进行分析，然后分解为树状结构。在渲染过程中，我们使用用户' s 当前速度来选择树模型并通过广度优先搜索和逆小波变换合成新的声音。我们进行了一项用户研究，以评估我们的虚拟声音的真实性及其在同时存在真实触觉提示的情况下对人类对纹理尺寸感知的影响。结果表明，在存在触觉提示的情况下，虚拟声音比单独的触觉提示可以更完整地捕捉纹理的粗糙度和硬度。然而，对滑溜的感知主要取决于触觉。与单独的触觉提示相比，虚拟声音可以更完整地捕捉纹理的粗糙度和硬度。然而，对滑溜的感知主要取决于触觉。与单独的触觉提示相比，虚拟声音可以更完整地捕捉纹理的粗糙度和硬度。然而，对滑溜的感知主要取决于触觉。