We introduce VA-Point-MVSNet, a novel visibility-aware point-based deep framework for multi-view stereo (MVS). Distinct from existing cost volume approaches, our method directly processes the target scene as point clouds. More specifically, our method predicts the depth in a coarse-to-fine manner. We first generate a coarse depth map, convert it into a point cloud and refine the point cloud iteratively by estimating the residual between the depth of the current iteration and that of the ground truth. Our network leverages 3D geometry priors and 2D texture information jointly and effectively by fusing them into a feature-augmented point cloud, and processes the point cloud to estimate the 3D flow for each point. This point-based architecture allows higher accuracy, more computational efficiency and more flexibility than cost-volume-based counterparts. Furthermore, our visibility-aware multi-view feature aggregation allows the network to aggregate multi-view appearance cues while taking into account visibility. Experimental results show that our approach achieves a significant improvement in reconstruction quality compared with state-of-the-art methods on the DTU and the Tanks and Temples dataset. The code of VA-Point-MVSNet proposed in this work will be released at https://github.com/callmeray/PointMVSNet .

中文翻译：

---

可见性感知点的多视角立体网络

我们介绍了 VA-Point-MVSNet，这是一种用于多视图立体 (MVS) 的新颖的基于可见性感知点的深度框架。与现有的成本量方法不同，我们的方法直接将目标场景处理为点云。更具体地说，我们的方法以粗到细的方式预测深度。我们首先生成一个粗略的深度图，将其转换为点云，并通过估计当前迭代深度与地面实况深度之间的残差来迭代地细化点云。我们的网络通过将 3D 几何先验和 2D 纹理信息融合到一个特征增强的点云中来联合有效地利用 3D 几何先验和 2D 纹理信息，并处理点云以估计每个点的 3D 流。这种基于点的架构允许更高的精度，比基于成本量的同行更高的计算效率和更大的灵活性。此外，我们的可见性感知多视图特征聚合允许网络在考虑可见性的同时聚合多视图外观线索。实验结果表明，与 DTU 和 Tanks and Temples 数据集上的最新方法相比，我们的方法在重建质量上取得了显着提高。本工作中提出的 VA-Point-MVSNet 的代码将在 实验结果表明，与 DTU 和 Tanks and Temples 数据集上的最新方法相比，我们的方法在重建质量上取得了显着提高。本工作中提出的 VA-Point-MVSNet 的代码将在 实验结果表明，与 DTU 和 Tanks and Temples 数据集上的最新方法相比，我们的方法在重建质量上取得了显着提高。本工作中提出的 VA-Point-MVSNet 的代码将在https://github.com/callmeray/PointMVSNet .