Light field cameras can capture the radiance and direction of light rays by a single exposure, providing a new perspective to photography and 3D geometry perception. However, existing sub-aperture based light field cameras are limited by their sensor resolution to obtain high spatial and angular resolution images simultaneously. In this paper, we propose an inference-reconstruction variational autoencoder (IR-VAE) to reconstruct a dense light field image out of four corner reference views in a light field image. The proposed IR-VAE is comprised of one inference network and one reconstruction network, where the inference network infers novel views from existing reference views and viewpoint conditions, and the reconstruction network reconstructs novel views from a latent variable that contains the information of reference views, novel views, and viewpoints. The conditional latent variable in the inference network is regularized by the latent variable in the reconstruction network to facilitate information flow between the conditional latent variable and novel views. We also propose a statistic distance measurement dubbed the mean local maximum mean discrepancy (MLMMD) to enable the measurement of the statistic distance between two distributions with high-resolution latent variables, which can capture richer information than their low-resolution counterparts. Finally, we propose a viewpoint-dependent indirect view synthesis method to synthesize novel views more efficiently by leveraging adaptive convolution. Experimental results show that our proposed methods outperform state-of-the-art methods on different light field datasets.

中文翻译：

---

用于光场图像重建的推理重建变分自动编码器


光场相机可以通过单次曝光捕获光线的亮度和方向，为摄影和 3D 几何感知提供新的视角。然而，现有的基于子孔径的光场相机受到其传感器分辨率的限制，无法同时获得高空间和角分辨率图像。在本文中，我们提出了一种推理重建变分自动编码器（IR-VAE），用于从光场图像中的四个角参考视图重建密集光场图像。所提出的 IR-VAE 由一个推理网络和一个重建网络组成，其中推理网络根据现有参考视图和视点条件推断新视图，重建网络根据包含参考视图信息的潜在变量重建新视图，新颖的观点和观点。推理网络中的条件潜变量由重建网络中的潜变量进行正则化，以促进条件潜变量和新颖视图之间的信息流。我们还提出了一种称为平均局部最大平均差异（MLMMD）的统计距离测量，以能够测量具有高分辨率潜在变量的两个分布之间的统计距离，与低分辨率对应物相比，它可以捕获更丰富的信息。最后，我们提出了一种依赖于视点的间接视图合成方法，通过利用自适应卷积更有效地合成新视图。实验结果表明，我们提出的方法在不同的光场数据集上优于最先进的方法。