当前位置: X-MOL 学术IEEE Trans. Vis. Comput. Graph. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Physically-inspired Deep Light Estimation from a Homogeneous-Material Object for Mixed Reality Lighting.
IEEE Transactions on Visualization and Computer Graphics ( IF 5.2 ) Pub Date : 2020-02-13 , DOI: 10.1109/tvcg.2020.2973050
Jinwoo Park , Hunmin Park , Sung-Eui Yoon , Woontack Woo

In mixed reality (MR), augmenting virtual objects consistently with real-world illumination is one of the key factors that provide a realistic and immersive user experience. For this purpose, we propose a novel deep learning-based method to estimate high dynamic range (HDR) illumination from a single RGB image of a reference object. To obtain illumination of a current scene, previous approaches inserted a special camera in that scene, which may interfere with user's immersion, or they analyzed reflected radiances from a passive light probe with a specific type of materials or a known shape. The proposed method does not require any additional gadgets or strong prior cues, and aims to predict illumination from a single image of an observed object with a wide range of homogeneous materials and shapes. To effectively solve this ill-posed inverse rendering problem, three sequential deep neural networks are employed based on a physically-inspired design. These networks perform end-to-end regression to gradually decrease dependency on the material and shape. To cover various conditions, the proposed networks are trained on a large synthetic dataset generated by physically-based rendering. Finally, the reconstructed HDR illumination enables realistic image-based lighting of virtual objects in MR. Experimental results demonstrate the effectiveness of this approach compared against state-of-the-art methods. The paper also suggests some interesting MR applications in indoor and outdoor scenes.

中文翻译:

混合现实照明的均质物体的物理启发式深光估计。

在混合现实(MR)中,以真实世界的照明一致地增强虚拟对象是提供逼真的沉浸式用户体验的关键因素之一。为此,我们提出了一种基于深度学习的新颖方法,可以从参考对象的单个RGB图像估计高动态范围(HDR)照明。为了获得当前场景的照明,以前的方法在该场景中插入了一个特殊的摄像头,这可能会干扰用户的沉浸感,或者他们使用特定类型的材料或已知形状来分析无源光探测器的反射辐射。所提出的方法不需要任何其他小工具或强烈的先验提示,并且旨在根据具有广泛范围的均质材料和形状的被观察物体的单个图像来预测照明。为了有效解决这一不适定的逆渲染问题,基于物理灵感设计,采用了三个顺序的深度神经网络。这些网络执行端到端回归以逐渐减少对材料和形状的依赖性。为了涵盖各种条件,在通过基于物理的渲染生成的大型综合数据集上训练了建议的网络。最后,重建的HDR照明可实现MR中虚拟对象的基于图像的逼真的照明。实验结果证明了该方法与最新技术方法相比的有效性。本文还提出了一些有趣的MR在室内和室外场景中的应用。这些网络执行端到端回归以逐渐减少对材料和形状的依赖性。为了涵盖各种条件,建议的网络在通过基于物理的渲染生成的大型合成数据集上进行训练。最后,重建的HDR照明可实现MR中虚拟对象的基于图像的逼真的照明。实验结果证明了该方法与最新技术方法相比的有效性。本文还提出了一些有趣的MR在室内和室外场景中的应用。这些网络执行端到端回归以逐渐减少对材料和形状的依赖性。为了涵盖各种条件,建议的网络在通过基于物理的渲染生成的大型合成数据集上进行训练。最后,重建的HDR照明可实现MR中虚拟对象的基于图像的逼真的照明。实验结果证明了该方法与最新技术方法相比的有效性。本文还提出了一些有趣的MR在室内和室外场景中的应用。实验结果证明了该方法与最新技术方法相比的有效性。本文还提出了一些有趣的MR在室内和室外场景中的应用。实验结果证明了该方法与最新技术方法相比的有效性。本文还提出了一些有趣的MR在室内和室外场景中的应用。
更新日期:2020-04-22
down
wechat
bug