Context‐Aware Mixed Reality: A Learning‐Based Framework for Semantic‐Level Interaction,Computer Graphics Forum

当前位置： X-MOL 学术 › Comput. Graph. Forum › 论文详情

Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)

Context‐Aware Mixed Reality: A Learning‐Based Framework for Semantic‐Level Interaction
Computer Graphics Forum ( IF 2.7 ) Pub Date : 2019-11-14 , DOI: 10.1111/cgf.13887
L. Chen ₁ , W. Tang ₁ , N. W. John ₂ , T. R. Wan ₃ , J. J. Zhang ₄

Affiliation

Mixed reality (MR) is a powerful interactive technology for new types of user experience. We present a semantic‐based interactive MR framework that is beyond current geometry‐based approaches, offering a step change in generating high‐level context‐aware interactions. Our key insight is that by building semantic understanding in MR, we can develop a system that not only greatly enhances user experience through object‐specific behaviours, but also it paves the way for solving complex interaction design challenges. In this paper, our proposed framework generates semantic properties of the real‐world environment through a dense scene reconstruction and deep image understanding scheme. We demonstrate our approach by developing a material‐aware prototype system for context‐aware physical interactions between the real and virtual objects. Quantitative and qualitative evaluation results show that the framework delivers accurate and consistent semantic information in an interactive MR environment, providing effective real‐time semantic‐level interactions.

中文翻译：

上下文感知混合现实：基于学习的语义级交互框架

混合现实 (MR) 是一种用于新型用户体验的强大交互技术。我们提出了一个基于语义的交互式 MR 框架，它超越了当前基于几何的方法，在生成高级上下文感知交互方面提供了一个步骤变化。我们的主要见解是，通过在 MR 中建立语义理解，我们可以开发一个系统，该系统不仅可以通过特定于对象的行为极大地增强用户体验，而且还可以为解决复杂的交互设计挑战铺平道路。在本文中，我们提出的框架通过密集场景重建和深度图像理解方案生成现实世界环境的语义属性。我们通过开发用于真实和虚拟对象之间的上下文感知物理交互的材料感知原型系统来演示我们的方法。

更新日期：2019-11-14

点击分享查看原文

点击收藏

公开下载

阅读更多本刊最新论文本刊介绍/投稿指南11