当前位置: X-MOL 学术Adv. Funct. Mater. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Functional Materials and Devices for XR (VR/AR/MR) Applications
Advanced Functional Materials ( IF 19.0 ) Pub Date : 2021-09-24 , DOI: 10.1002/adfm.202106546
Seung Hwan Ko 1 , John Rogers 2
Affiliation  

The “Metaverse” (meaning “beyond universe”) is the next big subject in the online world and it is essentially a shared virtual space in which virtual life and reality interact, co-exist, evolve together, and various cultural, social, and economic activities take place within them to create value. It combines an enhanced physical reality with a virtual space to encompass them with special technology; XR (eXtended Reality) which includes VR (Virtual Reality), AR (Augmented Reality) and MR (Mixed Reality). Steven Spielberg's recent science fiction action-adventure film, “Ready Player One” (2018) well illustrated a possible future metaverse society. Beyond this science fiction vision of the future, we are already entering the era of the metaverse, where the current COVID-19 pandemic situation and the associated demands for remote forms of social interactions are accelerating factors.

The technical issues in XR (VR+AR+MR) lie in development of the software and the hardware system as well. We are currently witnessing a remarkable development in XR software engineering, while XR devices and materials development remain in the very infancy stage. XR hardware systems are expected to artificially reconstruct various human feelings and transmit the information to the human sensory systems to make them believe they are really feeling the virtual world. Oculus (Facebook), Vive (HTC), Index (Valve) are the popular commercial VR devices, but they are limited to specific feelings originating mainly from visual and auditory cues and realized only in rigid forms of materials and devices. Compared with visual and auditory senses, less advancement has been made in tactile senses (haptics) including temperature, texture, pressure, and touch. At the same time, rigid material and devices deteriorate the immersion in haptic feedback systems because intimate contact between haptic device and skin is very important to replicate the faithful haptic feedback.

By introducing new soft skin-like materials and devices to the XR research field, various next-generation haptic feedback systems are under development. Among various tactile senses, vibro-haptic (mechanical feeling by vibration) and thermo-haptic (thermal feeling by heating or cooling) are being studied. Vibro-haptic relies on the electrical/mechanical components to stimulate the skin to deliver sensations of physical touch in the virtual world. Rogers and colleagues (article number 2008805) review the full range of research activities in skin integrated vibro-haptic interfaces. Shea and colleagues (article number 2006639) demonstrate feel-through haptics that are ultra-thin and soft for more faithful feeling delivery directly to the skin with sufficient force to make virtual objects feel tangible, or to change the perceived texture of a physical object. In addition to the mechanical sensation, thermal sensation is very important to generate a more realistic artificial feeling in XR since heat carries fluent information on the surrounding environment. Ko and colleagues (article number 2007376) and Lee and colleagues (article number 2007952) review the fundamental mechanism, design strategies, and the rational guidelines for the adoption of thermo-haptic technology in XR application.

Multifunctional and high-resolution sensors are another integral part of natural and continuous interactions between human and XR devices. Especially, skin-compatible soft sensor development is important for implementing the user comfort and immersive feeling of feedback. Yeo and colleagues discuss packaging strategies, specific device designs and physiological sensors for XR application (article number 2005692). Javey and colleagues (article number 2008087) and Takei and colleagues (article number 2007436) review the important factors for materials and structures of wearable sensors, and their power sources and data communication. Skin electronics are one of the most promising platforms for sensing and actuation in future VR/AR devices. The detailed engineering aspects of e-skin devices such as input/output device, energy source and integrated systems are discussed by Someya and colleagues (article number 2009602). In addition to elastomer material, fiber and fiber assembly-based sensors are discussed for interactive textile electronics for XR application by Tao and colleagues (article number 2007254). Two new types of e-skin type sensors for XR application are demonstrated by Makarov and colleagues (article number 2007788) for a magnetosensitive E-Skin type sensor that can track motion and orientation in 3D and by Tee and colleagues (article number 2008650) for a predictive materials design system consisting of a capacitive micropyramidal structure sensor and its augmented reality application in surgical training.

The haptic feedback and sensor technology can be integrated to make more complex XR devices. Bae and colleagues (article number 2007772) develop a new multimodal sensor with a printed liquid metal circuit and two modes of haptic feedback (vibro-haptic + thermo-haptic) device is demonstrated to deliver temperature and tactile feeling in virtual reality. Majidi and colleagues (article number 2007428) discuss soft materials based on-body devices for wearable sensors and haptic feedback, as well as their current challenges and future perspective.

Robotic research is very closely related to XR technology such as robotic surgery or soft robotic teleoperation. Shepherd and colleagues (article number 2009364) review the recent advances in soft actuators and sensors for application in haptic feedback and sensing for soft robotics in XR application. The interface between machine and user is discussed by Chen and colleagues (article number 2008807) for the fusion of emerging stretchable electronics and machine learning technology and by Cheng and Colleagues (article number 2008347) for nanowire-based soft wearable HMI sensors and haptic feedback systems. Soft actuators for soft robotics are reviewed by Yi and colleagues (article number 2009835) for liquid crystal soft actuators for mixed reality applications and by Liu and colleagues (article number 2007437) for soft multifunctional tensile and torsional actuators.

Optical components are also important for head mound display. New characteristics of image sensors and displays for developing future XR systems such as transparent or deformable display are discussed (article number 2009281). Lee and colleagues (article number 2104105) present a study on azopolymeric optical Fourier surfaces for AR.

Lastly, three extensive reviews on the material for haptics technology are presented for polymeric material design and chemistry by Lipomi and colleagues (article number 2008375), the current trends in and perspectives for active materials in haptic technology by Kim and colleagues (article number 2008831), and haptic perception, mechanics, and material technologies for VR by Visell and colleagues (article number 2008186).

As with radical advances in any new technology, ethical issues must be considered. Although such topics lie outside of the scope of a journal like Advanced Functional Materials, in this editorial, we provide some comments. As James Spiegel[1] discussed, XR technologies have the following potential hazards: “First, XR poses potential mental health risks, including depersonalization/derealization disorder. Second, XR technology raises serious concerns related to personal neglect of users’ own actual bodies and real physical environments. Third, XR technologies may be used to record personal data which could be deployed in ways that threaten personal privacy and present a danger related to manipulation of users’ beliefs, emotions, and behaviors. Finally, there are other moral and social risks associated with the way XR blurs the distinction between the real and illusory.” Beyond the excitement around technological advancements in functional materials and devices for XR applications, these issues should be kept in mind in the process of XR development.

We believe that this collection of research articles and reviews provides a thorough outline of the recent progress, challenges, and future opportunities in functional materials and devices for XR (VR/AR/MR) and, furthermore, that this issue of the journal can serve as a guideline and inspiration to current and next generation researchers in this rapidly expanding field.



中文翻译:

用于 XR (VR/AR/MR) 应用的功能材料和设备

“元界”(意为“超越宇宙”)是网络世界的下一个大主题,它本质上是一个虚拟生活与现实互动、共存、共同进化的共享虚拟空间,各种文化、社会和经济活动在它们内部发生以创造价值。它将增强的物理现实与虚拟空间相结合,以特殊技术将它们包围起来;XR(E X倾向于ř eality),其包括VR(V irtual ř eality),AR(ugmented ř eality)和MR(中号ixed ř现实)。史蒂文·斯皮尔伯格 (Steven Spielberg) 最近的科幻动作冒险电影《玩家一号》(Ready Player One)(2018 年)很好地说明了未来可能出现的元节社会。除了这种对未来的科幻想象之外,我们已经进入了元宇宙时代,当前的 COVID-19 大流行情况和对远程社交互动形式的相关需求是加速因素。

XR(VR+AR+MR)的技术问题还在于软件和硬件系统的开发。我们目前正在见证 XR 软件工程的显着发展,而 XR 设备和材料的开发仍处于起步阶段。XR 硬件系统有望人为地重建各种人类的感受,并将信息传输到人类的感官系统,使他们相信自己真的在感受虚拟世界。Oculus (Facebook)、Vive (HTC)、Index (Valve) 是流行的商业 VR 设备,但它们仅限于主要源自视觉和听觉线索的特定感觉,并且只能通过材料和设备的刚性形式实现。与视觉和听觉相比,触觉(触觉)的进步较小,包括温度、质地、压力和触觉。

通过将新的类软材料和设备引入 XR 研究领域,各种下一代触觉反馈系统正在开发中。在各种触觉中,正在研究振动触觉(通过振动产生的机械感觉)和热触觉(通过加热或冷却产生的热感觉)。Vibro-haptic 依靠电气/机械组件来刺激皮肤,从而在虚拟世界中提供身体接触的感觉。Rogers 及其同事(文章编号2008805)回顾了皮肤集成振动触觉界面的全方位研究活动。Shea 及其同事(文章编号2006639) 展示超薄和柔软的触感触觉,以更忠实地将感觉直接传递到皮肤,并有足够的力量使虚拟物体感觉有形,或改变物理物体的感知纹理。除了机械感觉,热感觉对于在 XR 中产生更逼真的人工感觉非常重要,因为热量会携带有关周围环境的流畅信息。Ko 及其同事(文章编号2007376)和 Lee 及其同事(文章编号2007952)回顾了在 XR 应用中采用热触觉技术的基本机制、设计策略和合理指南。

多功能和高分辨率传感器是人类和 XR 设备之间自然和持续交互的另一个组成部分。特别是,皮肤兼容软传感器的开发对于实现用户舒适度和身临其境的反馈感觉非常重要。Yeo 及其同事讨论了用于 XR 应用的封装策略、特定设备设计和生理传感器(文章编号2005692)。Javey 及其同事(文章编号2008087)和 Takei 及其同事(文章编号2007436)回顾可穿戴传感器的材料和结构的重要因素,以及它们的电源和数据通信。皮肤电子设备是未来 VR/AR 设备中最有前途的传感和驱动平台之一。Someya 及其同事讨论了电子皮肤设备的详细工程方面,例如输入/输出设备、能源和集成系统(文章编号2009602)。除了弹性体材料外,Tao 及其同事还讨论了基于纤维和纤维组件的传感器用于 XR 应用的交互式纺织电子产品(文章编号2007254)。Makarov 及其同事展示了两种用于 XR 应用的新型电子皮肤型传感器(文章编号2007788) 用于可跟踪 3D 运动和方向的磁敏E-Skin 型传感器,Tee 及其同事(文章编号2008650)用于预测材料设计系统,该系统由电容微锥体结构传感器及其在手术训练中的增强现实应用组成。

触觉反馈和传感器技术可以集成以制造更复杂的 XR 设备。Bae 及其同事(文章编号2007772)开发了一种具有印刷液态金属电路和两种触觉反馈模式(振动触觉 + 热触觉)设备的新型多模态传感器,该设备被证明可以在虚拟现实中提供温度和触觉。Majidi 及其同事(文章编号2007428)讨论了用于可穿戴传感器和触觉反馈的基于身体设备的软材料,以及他们当前的挑战和未来前景。

机器人研究与机器人手术或软机器人遥操作等 XR 技术密切相关。Shepherd 及其同事(文章编号2009364)回顾了软执行器和传感器在 XR 应用中软机器人触觉反馈和传感方面的最新进展。Chen 及其同事(文章编号2008807)讨论了机器和用户之间的接口,以融合新兴的可拉伸电子学和机器学习技术;Cheng 及其同事(文章编号2008347)讨论了基于纳米线的软可穿戴 HMI 传感器和触觉反馈系统. Yi 及其同事审查了用于软机器人的软执行器(文章编号2009835) 用于混合现实应用的液晶软致动器,Liu 及其同事(文章编号2007437)用于软多功能拉伸和扭转致动器。

光学元件对于头丘显示器也很重要。讨论了用于开发未来 XR 系统(例如透明或可变形显示器)的图像传感器和显示器的新特性(文章编号2009281)。Lee 及其同事(文章编号2104105)提出了一项关于 AR 的偶氮聚合物光学傅立叶表面的研究。

最后,Lipomi 及其同事(文章编号2008375)针对用于聚合物材料设计和化学的触觉技术材料进行了三篇广泛的评论,Kim 及其同事(文章编号2008831)介绍了触觉技术中活性材料的当前趋势和前景以及Visell及其同事的 VR 触觉感知、力学和材料技术(文章编号2008186)。

与任何新技术的根本进步一样,必须考虑道德问题。尽管此类主题不在Advanced Functional Materials等期刊的范围内,但在本社论中,我们提供了一些评论。正如詹姆斯·斯皮格尔[ 1 ]讨论中,XR 技术具有以下潜在危害:“首先,XR 会带来潜在的心理健康风险,包括人格解体/现实解体障碍。其次,XR 技术引发了与个人忽视用户自身真实身体和真实物理环境相关的严重担忧。第三,XR 技术可用于记录个人数据,这些数据可能以威胁个人隐私的方式部署,并存在与操纵用户信仰、情绪和行为相关的危险。最后,XR 模糊真实与虚幻之间的区别还存在其他道德和社会风险。” 除了围绕 XR 应用的功能材料和设备的技术进步令人兴奋之外,在 XR 开发过程中还应牢记这些问题。

我们相信,这本研究文章和评论集全面概述了 XR (VR/AR/MR) 功能材料和设备的最新进展、挑战和未来机遇,此外,本期期刊可以服务于作为这一快速发展领域中当前和下一代研究人员的指导方针和灵感。

更新日期:2021-09-24
down
wechat
bug