Image and Vision Computing ( IF 4.2 ) Pub Date : 2020-01-14 , DOI: 10.1016/j.imavis.2020.103874 Wenhui Zhou , Gaomin Liu , Jiangwei Shi , Hua Zhang , Guojun Dai
Light field imaging has recently become a promising technology for 3D rendering and displaying. However, capturing real-world light field images still faces many challenges in both the quantity and quality. In this paper, we develop a learning based technique to reconstruct light field from a single 2D RGB image. It includes three steps: unsupervised monocular depth estimation, view synthesis and depth-guided view inpainting. We first propose a novel monocular depth estimation network to predict disparity maps of each sub-aperture views from the central view of light field. Then we synthesize the initial sub-aperture views by using the warping scheme. Considering that occlusion makes synthesis ambiguous for pixels invisible in the central view, we present a simple but effective fully convolutional network (FCN) for view inpainting. Note that the proposed network architecture is a general framework for light field reconstruction, which can be extended to take a sparse set of views as input without changing any structure or parameters of the network. Comparison experiments demonstrate that our method outperforms the state-of-the-art light field reconstruction methods with single-view input, and achieves comparable results with the multi-input methods.
中文翻译:
深度引导视图合成,可从单个图像重建光场
光场成像最近已成为3D渲染和显示的有前途的技术。但是,捕获现实世界的光场图像在数量和质量上仍然面临许多挑战。在本文中,我们开发了一种基于学习的技术来从单个2D RGB图像重建光场。它包括三个步骤:无监督单眼深度估计,视图合成和深度引导视图修补。我们首先提出一种新颖的单眼深度估计网络,以从光场的中心视图预测每个子孔径视图的视差图。然后,我们使用变形方案合成初始子孔径视图。考虑到遮挡使像素在中央视图中不可见,因此合成模棱两可,因此我们提出了一种简单但有效的全卷积网络(FCN)用于视图修复。注意,提出的网络体系结构是用于光场重建的通用框架,可以扩展为将稀疏的视图集作为输入,而无需更改网络的任何结构或参数。对比实验表明,我们的方法优于单视图输入的最新光场重建方法,并且与多输入方法可比。