Light fields present a rich way to represent the 3D world by capturing the spatio-angular dimensions of the visual signal. However, the popular way of capturing light fields (LF) via a plenoptic camera presents a spatio-angular resolution trade-off. To address this issue, computational imaging techniques such as compressive light field and programmable coded aperture have been proposed, which reconstruct full sensor resolution LF from coded projections of the LF. Here, we present a unified learning framework that can reconstruct LF from a variety of multiplexing schemes with minimal number of coded images as input. We consider three light field capture schemes: heterodyne capture scheme with code placed near the sensor, coded aperture scheme with code at the camera aperture and finally the dual exposure scheme of capturing a focus-defocus pair where there is no explicit coding. Our algorithm consists of three stages: Firstly, we recover the all-in-focus image from the coded image. Secondly, we estimate the disparity maps for all the LF views from the coded image and the all-in-focus image. And finally, we render the LF by warping the all-in-focus image using the estimated disparity maps. We show that our proposed learning algorithm performs either on par with or better than the state-of-the-art methods for all the three multiplexing schemes. LF from focus-defocus pair is especially attractive as it requires no hardware modification and produces LF reconstructions that are comparable to the current state of the art learning-based view synthesis approaches from multiple images. Thus, our work paves the way for capturing full-resolution LF using conventional cameras such as DSLRs and smartphones.

中文翻译：

---

基于统一学习的编码投影光场重建框架

光场通过捕获视觉信号的空间角度维度，提供了一种丰富的方式来表示 3D 世界。然而，通过全光相机捕捉光场 (LF) 的流行方式呈现出空间角分辨率的权衡。为了解决这个问题，已经提出了诸如压缩光场和可编程编码孔径之类的计算成像技术，它们从LF的编码投影重建全传感器分辨率LF。在这里，我们提出了一个统一的学习框架，该框架可以从各种多路复用方案中以最少数量的编码图像作为输入来重建 LF。我们考虑三种光场捕获方案：外差捕获方案，代码放置在传感器附近，编码光圈方案，在相机光圈处使用代码，最后是在没有明确编码的情况下捕获焦点-散焦对的双重曝光方案。我们的算法包括三个阶段：首先，我们从编码图像中恢复全焦图像。其次，我们从编码图像和全焦图像中估计所有 LF 视图的视差图。最后，我们通过使用估计的视差图扭曲全焦图像来渲染 LF。我们表明，对于所有三种复用方案，我们提出的学习算法的性能与最先进的方法相当或更好。来自焦点散焦对的 LF 特别有吸引力，因为它不需要硬件修改，并且可以产生与当前最先进的基于学习的多幅图像视图合成方法相媲美的 LF 重建。因此，我们的工作为使用单反和智能手机等传统相机捕捉全分辨率低频铺平了道路。