We present a method to estimate an HDR environment map from a narrow field-of-view LDR camera image in real-time. This enables perceptually appealing reflections and shading on virtual objects of any material finish, from mirror to diffuse, rendered into a real physical environment using augmented reality. Our method is based on our efficient convolutional neural network architecture, EnvMapNet, trained end-to-end with two novel losses, ProjectionLoss for the generated image, and ClusterLoss for adversarial training. Through qualitative and quantitative comparison to state-of-the-art methods, we demonstrate that our algorithm reduces the directional error of estimated light sources by more than 50%, and achieves 3.7 times lower Frechet Inception Distance (FID). We further showcase a mobile application that is able to run our neural network model in under 9 ms on an iPhone XS, and render in real-time, visually coherent virtual objects in previously unseen real-world environments.

中文翻译：

---

实时增强现实的HDR环境地图估计

我们提出了一种从狭窄的视野LDR摄像机图像实时估计HDR环境图的方法。这样就可以使用增强现实技术，对从镜子到漫反射的任何材质完成的虚拟对象（从镜子到漫反射）在感知上吸引人的反射和阴影。我们的方法基于我们高效的卷积神经网络架构EnvMapNet，它接受了端到端的两个新损失训练，即生成图像的ProjectionLoss和对抗训练的ClusterLoss。通过与最先进方法的定性和定量比较，我们证明了我们的算法将估计光源的方向误差降低了50％以上，并实现了3.7倍的Frechet起始距离（FID）降低。