We introduce a novel learning-based method to reconstruct the high-quality geometry and complex, spatially-varying BRDF of an arbitrary object from a sparse set of only six images captured by wide-baseline cameras under collocated point lighting. We first estimate per-view depth maps using a deep multi-view stereo network; these depth maps are used to coarsely align the different views. We propose a novel multi-view reflectance estimation network architecture that is trained to pool features from these coarsely aligned images and predict per-view spatially-varying diffuse albedo, surface normals, specular roughness and specular albedo. We do this by jointly optimizing the latent space of our multi-view reflectance network to minimize the photometric error between images rendered with our predictions and the input images. While previous state-of-the-art methods fail on such sparse acquisition setups, we demonstrate, via extensive experiments on synthetic and real data, that our method produces high-quality reconstructions that can be used to render photorealistic images.

中文翻译：

---

深度 3D 捕获：来自稀疏多视图图像的几何和反射

我们引入了一种新的基于学习的方法，用于从宽基线相机在并置点照明下捕获的仅六个图像的稀疏集合中重建任意对象的高质量几何形状和复杂的、空间变化的 BRDF。我们首先使用深度多视图立体网络估计每个视图的深度图；这些深度图用于粗略地对齐不同的视图。我们提出了一种新颖的多视图反射估计网络架构，该架构经过训练以汇集这些粗对齐图像的特征，并预测每个视图空间变化的漫反射、表面法线、镜面粗糙度和镜面反射率。我们通过联合优化我们的多视图反射网络的潜在空间来最小化我们的预测渲染的图像和输入图像之间的光度误差来做到这一点。