This paper is concerned with the problem of multi-view 3D reconstruction with an un-calibrated micro-lens array based light field camera. To acquire 3D Euclidean reconstruction, existing approaches commonly apply the calibration with a checkerboard and motion estimation from static scenes in two steps. Self-calibration is the process of simultaneously estimating intrinsic and extrinsic parameters directly from un-calibrated light fields without the help of a checkerboard. While the self-calibration technique for conventional (pinhole) camera is well understood, how to extend it to light field camera remains a challenging task. This is primarily due to the ultra-small baseline of the light field camera. We propose an effective self-calibration method for a light field camera for automatic metric reconstruction without a laborious pre-calibration process. In contrast to conventional self-calibration, we show how such a self-calibration method can be made numerically stable, by exploiting the regularity and measurement redundancies unique for the light field camera. The proposed method is built upon the derivation of a novel ray-space homography constraint (RSHC) using Plücker parameterization as well as a ray-space infinity homography (RSIH). We also propose a new concept of “rays of the absolute conic (RAC)” defined as a special quadric in 5D projective space \({\mathbb {P}}^5\). A set of new equations are established and solved for self-calibration and 3D metric reconstruction specifically designed for a light field camera . We validate the efficacy of the proposed method on both synthetic and real light fields, and have obtained superior results in both accuracy and robustness.

本文关注的是使用未校准的基于微透镜阵列的光场相机进行多视图 3D 重建的问题。为了获得 3D 欧几里德重建，现有方法通常分两步应用棋盘校准和静态场景的运动估计。自校准是直接从未校准的光场同时估计内在和外在参数的过程，无需棋盘的帮助。虽然传统（针孔）相机的自校准技术已广为人知，但如何将其扩展到光场相机仍然是一项具有挑战性的任务。这主要是由于光场相机的超小基线。我们为光场相机提出了一种有效的自校准方法，用于自动度量重建，无需费力的预校准过程。与传统的自校准相比，我们展示了如何通过利用光场相机独有的规律性和测量冗余来使这种自校准方法在数值上稳定。所提出的方法建立在使用 Plücker 参数化以及射线空间无限单应性 (RSIH) 的新型射线空间单应性约束 (RSHC) 的推导之上。我们还提出了一个新概念“绝对圆锥曲线 (RAC)”，定义为 5D 投影空间中的特殊二次曲面 通过利用光场相机独有的规律性和测量冗余。所提出的方法建立在使用 Plücker 参数化以及射线空间无限单应性 (RSIH) 的新型射线空间单应性约束 (RSHC) 的推导之上。我们还提出了一个新概念“绝对圆锥曲线 (RAC)”，定义为 5D 投影空间中的特殊二次曲面 通过利用光场相机独有的规律性和测量冗余。所提出的方法建立在使用 Plücker 参数化以及射线空间无限单应性 (RSIH) 的新型射线空间单应性约束 (RSHC) 的推导之上。我们还提出了一个新概念“绝对圆锥曲线 (RAC)”，定义为 5D 投影空间中的特殊二次曲面\({\mathbb {P}}^5\)。建立并求解了一组新方程，用于专门为光场相机设计的自校准和 3D 度量重建。我们验证了所提出的方法在合成光场和真实光场上的有效性，并在准确性和鲁棒性方面获得了优异的结果。