Virtual reality has the potential to change the way we create and consume content in our everyday life. Entertainment, training, design and manufacturing, communication, or advertising are all applications that already benefit from this new medium reaching consumer level. VR is inherently different from traditional media: it offers a more immersive experience, and has the ability to elicit a sense of presence through the place and plausibility illusions. It also gives the user unprecedented capabilities to explore their environment, in contrast with traditional media. In VR, like in the real world, users integrate the multimodal sensory information they receive to create a unified perception of the virtual world. Therefore, the sensory cues that are available in a virtual environment can be leveraged to enhance the final experience. This may include increasing realism, or the sense of presence; predicting or guiding the attention of the user through the experience; or increasing their performance if the experience involves the completion of certain tasks. In this state-of-the-art report, we survey the body of work addressing multimodality in virtual reality, its role and benefits in the final user experience. The works here reviewed thus encompass several fields of research, including computer graphics, human computer interaction, or psychology and perception. Additionally, we give an overview of different applications that leverage multimodal input in areas such as medicine, training and education, or entertainment; we include works in which the integration of multiple sensory information yields significant improvements, demonstrating how multimodality can play a fundamental role in the way VR systems are designed, and VR experiences created and consumed.

中文翻译：

---

VR中的多模式性：一项调查

虚拟现实有可能改变我们在日常生活中创建和消费内容的方式。娱乐，培训，设计与制造，通信或广告都是已经从这种达到消费者水平的新媒体中受益的应用程序。VR与传统媒体在本质上有所不同：VR提供了更加身临其境的体验，并具有通过地点和似是而非的幻觉引起人们临场感的能力。与传统媒体相比，它还为用户提供了探索其环境的空前功能。在VR中，就像在现实世界中一样，用户将收到的多模式感官信息整合在一起，以创建对虚拟世界的统一感知。因此，可以利用虚拟环境中可用的感官提示来增强最终体验。这可能包括增加现实主义或存在感；通过体验预测或引导用户的注意力；或如果经验涉及完成某些任务，则提高他们的绩效。在此最新报告中，我们调查了解决虚拟现实中多模式问题的工作主体，其在最终用户体验中的作用和益处。因此，这里审查的作品涵盖了多个研究领域，包括计算机图形学，人机交互或心理学和知觉。此外，我们概述了在医学，培训和教育或娱乐等领域利用多模式输入的不同应用。我们纳入的作品中，多种感官信息的整合产生了重大改进，