当前位置: X-MOL 学术IEEE Trans. Pattern Anal. Mach. Intell. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
SurRF: Unsupervised Multi-View Stereopsis by Learning Surface Radiance Field.
IEEE Transactions on Pattern Analysis and Machine Intelligence ( IF 23.6 ) Pub Date : 2022-10-04 , DOI: 10.1109/tpami.2021.3116695
Jinzhi Zhang , Mengqi Ji , Guangyu Wang , Zhiwei Xue , Shengjin Wang , Lu Fang

The recent success in supervised multi-view stereopsis (MVS) relies on the onerously collected real-world 3D data. While the latest differentiable rendering techniques enable unsupervised MVS, they are restricted to discretized (e.g., point cloud) or implicit geometric representation, suffering from either low integrity for a textureless region or less geometric details for complex scenes. In this paper, we propose SurRF, an unsupervised MVS pipeline by learning Surface Radiance Field, i.e., a radiance field defined on a continuous and explicit 2D surface. Our key insight is that, in a local region, the explicit surface can be gradually deformed from a continuous initialization along view-dependent camera rays by differentiable rendering. That enables us to define the radiance field only on a 2D deformable surface rather than in a dense volume of 3D space, leading to compact representation while maintaining complete shape and realistic texture for large-scale complex scenes. We experimentally demonstrate that the proposed SurRF produces competitive results over the-state-of-the-art on various real-world challenging scenes, without any 3D supervision. Moreover, SurRF shows great potential in owning the joint advantages of mesh (scene manipulation), continuous surface (high geometric resolution), and radiance field (realistic rendering).

中文翻译:

SurRF:通过学习表面辐射场的无监督多视图立体视觉。

最近在有监督的多视图立体视觉 (MVS) 方面的成功依赖于繁重收集的真实世界 3D 数据。虽然最新的可微渲染技术支持无监督 MVS,但它们仅限于离散化(例如,点云)或隐式几何表示,无纹理区域的完整性低或复杂场景的几何细节较少。在本文中,我们提出了 SurRF,一种通过学习表面辐射场(即定义在连续且显式的 2D 表面上的辐射场)的无监督 MVS 管道。我们的关键见解是,在局部区域中,显式表面可以通过可微渲染沿着依赖于视图的相机光线从连续初始化逐渐变形。这使我们能够仅在 2D 可变形表面上而不是在密集的 3D 空间中定义辐射场,从而实现紧凑的表示,同时为大规模复杂场景保持完整的形状和逼真的纹理。我们通过实验证明,在没有任何 3D 监督的情况下,所提出的 SurRF 在各种现实世界具有挑战性的场景中产生了与最新技术相比具有竞争力的结果。此外,SurRF 在拥有网格(场景操作)、连续表面(高几何分辨率)和辐射场(逼真渲染)的联合优势方面显示出巨大的潜力。我们通过实验证明,在没有任何 3D 监督的情况下,所提出的 SurRF 在各种现实世界具有挑战性的场景中产生了与最新技术相比具有竞争力的结果。此外,SurRF 在拥有网格(场景操作)、连续表面(高几何分辨率)和辐射场(逼真渲染)的联合优势方面显示出巨大的潜力。我们通过实验证明,在没有任何 3D 监督的情况下,所提出的 SurRF 在各种现实世界具有挑战性的场景中产生了与最新技术相比具有竞争力的结果。此外,SurRF 在拥有网格(场景操作)、连续表面(高几何分辨率)和辐射场(逼真渲染)的联合优势方面显示出巨大的潜力。
更新日期:2021-09-30
down
wechat
bug