We propose a full study and methodology for multi-view stereo reconstruction with performance capture data. Multi-view 3D reconstruction has largely been studied with general, high resolution and high texture content inputs, where classic low-level feature extraction and matching are generally successful. However in performance capture scenarios, texture content is limited by wider angle shots resulting in smaller subject projection areas, and intrinsically low image content of casual clothing. We present a dedicated pipeline, based on a per-camera depth map sweeping strategy, analyzing in particular how recent deep network advances allow to replace classic multi-view photoconsistency functions with one that is learned. We show that learning based on a volumetric receptive field around a 3D depth candidate improves over using per-view 2D windows, giving the photoconsistency inference more visibility over local 3D correlations in viewpoint color aggregation. Despite being trained on a standard dataset of scanned static objects, the proposed method is shown to generalize and significantly outperform existing approaches on performance capture data, while achieving competitive results on recent benchmarks.

中文翻译：

---

体积扫描：学习多视图形状重建的光一致性

我们提出了使用性能捕获数据进行多视图立体重建的完整研究和方法。多视图 3D 重建已经在大量使用通用、高分辨率和高纹理内容输入的情况下进行了研究，其中经典的低级特征提取和匹配通常是成功的。然而，在表演捕捉场景中，纹理内容受到更广角度拍摄的限制，导致被摄体投影区域更小，休闲服装的图像内容本质上较低。我们提出了一个基于每相机深度图扫描策略的专用管道，特别分析了最近的深度网络进步如何允许用学习的函数替换经典的多视图光一致性函数。我们表明，基于 3D 深度候选周围的体积感受野的学习比使用每视图 2D 窗口有所改进，使光一致性推断在视点颜色聚合中对局部 3D 相关性具有更高的可见性。尽管在扫描静态对象的标准数据集上进行了训练，但所提出的方法被证明可以概括并显着优于现有的性能捕获数据方法，同时在最近的基准测试中取得了有竞争力的结果。