当前位置: X-MOL 学术Comput. Graph. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Snap2cad: 3D indoor environment reconstruction for AR/VR applications using a smartphone device
Computers & Graphics ( IF 2.5 ) Pub Date : 2021-07-24 , DOI: 10.1016/j.cag.2021.07.014
Alessandro Manni 1 , Damiano Oriti 1 , Andrea Sanna 1 , Francesco De Pace 1 , Federico Manuri 1
Affiliation  

Indoor environment reconstruction is a challenging task in Computer Vision and Computer Graphics, especially when Extended Reality (XR) technologies are considered. Current solutions that employ dedicated depth sensors require scanning of the environment and tend to suffer from low resolution and noise, whereas solutions that rely on a single photo of a scene cannot predict the actual position and scale of objects due to scale ambiguity. The proposed system addresses these limitations by allowing the user to capture single views of objects using an Android smartphone equipped with a single RGB camera and supported by Google ARCore. The system includes 1) an Android app tracking the smartphone’s position relative to the world, capturing a single RGB image for each object and estimating depth information of the scene, 2) a program running on a server that classifies the framed objects, retrieves the corresponding 3D models from a database and estimates their position, vertical rotation, and scale factor without deforming the shape. The system has been assessed measuring the translational, rotational and scaling errors of the considered objects with respect to the physical ones acting as a ground truth. The main outcomes show that the proposed solution obtains a maximum error of 18% for the scaling factor, less than nine centimeters for the position and less than 18 for the rotation. These results suggest that the proposed system can be employed for XR applications, thus bridging the gap between the real and virtual worlds.



中文翻译:

Snap2cad:使用智能手机设备为 AR/VR 应用程序重建 3D 室内环境

室内环境重建是计算机视觉和计算机图形学中的一项具有挑战性的任务,尤其是在考虑扩展现实 (XR) 技术时。当前采用专用深度传感器的解决方案需要对环境进行扫描,并且往往会受到低分辨率和噪声的影响,而依赖于场景的单张照片的解决方案由于尺度模糊而无法预测对象的实际位置和尺度。提议的系统通过允许用户使用配备单个 RGB 摄像头并由 Google ARCore 支持的 Android 智能手机捕获对象的单个视图来解决这些限制。该系统包括 1) 一个 Android 应用程序跟踪智能手机相对于世界的位置,为每个对象捕获单个 RGB 图像并估计场景的深度信息,2) 在服务器上运行的程序,它对框架对象进行分类,从数据库中检索相应的 3D 模型,并在不变形形状的情况下估计它们的位置、垂直旋转和比例因子。该系统已经过评估,可以测量所考虑对象相对于充当基本事实的物理对象的平移、旋转和缩放误差。主要结果表明,所提出的解决方案获得的比例因子最大误差为 18%,位置小于 9 厘米,小于 18 所考虑对象相对于作为基本事实的物理对象的旋转和缩放误差。主要结果表明,所提出的解决方案获得的比例因子最大误差为 18%,位置小于 9 厘米,小于 18 所考虑对象相对于作为基本事实的物理对象的旋转和缩放误差。主要结果表明,所提出的解决方案获得的比例因子最大误差为 18%,位置小于 9 厘米,小于 18为旋转。这些结果表明,所提出的系统可用于 XR 应用,从而弥合现实世界和虚拟世界之间的差距。

更新日期:2021-09-07
down
wechat
bug