In this study, we designed a four-dimensional (4D) audiovisual entertainment system called Sense. This system comprises a scene recognition system and hardware modules that provide haptic sensations for users when they watch movies and animations at home. In the scene recognition system, we used Google Cloud Vision to detect common scene elements in a video, such as fire, explosions, wind, and rain, and further determine whether the scene depicts hot weather, rain, or snow. Additionally, for animated videos, we applied deep learning with a single shot multibox detector to detect whether the animated video contained scenes of fire-related objects. The hardware module was designed to provide six types of haptic sensations set as line-symmetry to provide a better user experience. After the system considers the results of object detection via the scene recognition system, the system generates corresponding haptic sensations. The system integrates deep learning, auditory signals, and haptic sensations to provide an enhanced viewing experience.

中文翻译：

---

具有深度学习和触觉的桌面视听娱乐系统设计

在这项研究中，我们设计了一个名为 Sense 的四维 (4D) 视听娱乐系统。该系统包括场景识别系统和硬件模块，为用户在家看电影和动画时提供触觉。在场景识别系统中，我们使用谷歌云视觉检测视频中常见的场景元素，如火、爆炸、风和雨，并进一步判断场景是否描绘了炎热的天气、雨或雪。此外，对于动画视频，我们使用单发多框检测器应用深度​​学习来检测动画视频是否包含与火灾相关的物体场景。硬件模块旨在提供设置为线对称的六种类型的触觉，以提供更好的用户体验。系统通过场景识别系统考虑物体检测结果后，产生相应的触觉。该系统集成了深度学习、听觉信号和触觉，以提供增强的观看体验。