Depth, colour, and thermal images contain practical and actionable information for the blind. Conveying this information through alternative modalities such as audition creates new interaction possibilities for users as well as opportunities to study neuroplasticity. The ‘SoundSight’ App (www.SoundSight.co.uk) is a smartphone platform that allows 3D position, colour, and thermal information to directly control thousands of high-quality sounds in real-time to create completely unique and responsive soundscapes for the user. Users can select the specific sensor input and style of auditory output, which can be based on anything—tones, rainfall, speech, instruments, or even full musical tracks. Appropriate default settings for image-sonification are given by designers, but users still have a fine degree of control over the timing and selection of these sounds. Through utilising smartphone technology with a novel approach to sonification, the SoundSight App provides a cheap, widely accessible, scalable, and flexible sensory tool. In this paper we discuss common problems encountered with assistive sensory tools reaching long-term adoption, how our device seeks to address these problems, its theoretical background, its technical implementation, and finally we showcase both initial user experiences and a range of use case scenarios for scientists, artists, and the blind community.

深度、颜色和热图像包含可供盲人使用的实用信息。通过试听等替代方式传达这些信息为用户创造了新的交互可能性以及研究神经可塑性的机会。“SoundSight”应用程序 (www.SoundSight.co.uk) 是一个智能手机平台，它允许 3D 位置、颜色和热信息直接实时控制数千种高质量声音，从而为用户创建完全独特且响应迅速的音景。用户。用户可以选择特定的传感器输入和听觉输出风格，这可以基于任何东西——音调、降雨、语音、乐器，甚至完整的音乐曲目。适当的图像声化默认设置由设计师提供，但是用户仍然可以很好地控制这些声音的时间和选择。通过利用智能手机技术和一种新颖的声音处理方法，SoundSight 应用程序提供了一种廉价、可广泛访问、可扩展且灵活的感官工具。在本文中，我们讨论了长期采用的辅助感官工具遇到的常见问题、我们的设备如何寻求解决这些问题、其理论背景、技术实现，最后我们展示了初始用户体验和一系列用例场景为科学家、艺术家和盲人社区服务。