We present self-supervised geometric perception (SGP), the first general framework to learn a feature descriptor for correspondence matching without any ground-truth geometric model labels (e.g., camera poses, rigid transformations). Our first contribution is to formulate geometric perception as an optimization problem that jointly optimizes the feature descriptor and the geometric models given a large corpus of visual measurements (e.g., images, point clouds). Under this optimization formulation, we show that two important streams of research in vision, namely robust model fitting and deep feature learning, correspond to optimizing one block of the unknown variables while fixing the other block. This analysis naturally leads to our second contribution -- the SGP algorithm that performs alternating minimization to solve the joint optimization. SGP iteratively executes two meta-algorithms: a teacher that performs robust model fitting given learned features to generate geometric pseudo-labels, and a student that performs deep feature learning under noisy supervision of the pseudo-labels. As a third contribution, we apply SGP to two perception problems on large-scale real datasets, namely relative camera pose estimation on MegaDepth and point cloud registration on 3DMatch. We demonstrate that SGP achieves state-of-the-art performance that is on-par or superior to the supervised oracles trained using ground-truth labels.

中文翻译：

---

自我监督的几何感知

我们提出了自我监督的几何感知（SGP），这是第一个通用的框架，用于学习无需任何真实的几何模型标签（例如，相机姿势，刚性变换）进行对应匹配的特征描述符。我们的第一个贡献是将几何感知公式化为一个优化问题，可以在给定大量视觉测量（例如图像，点云）的情况下共同优化特征描述符和几何模型。在这种优化公式下，我们显示了视觉方面的两个重要研究流，即稳健的模型拟合和深度特征学习，对应于优化一个未知变量块，同时固定另一个块。这种分析自然导致了我们的第二个贡献-SGP算法执行交替最小化以解决联合优化。SGP迭代执行两个元算法：一位老师根据给定的学习特征进行健壮的模型拟合，以生成几何伪标签；另一名学生在嘈杂的伪标签监督下进行深度特征学习。作为第三贡献，我们将SGP应用于大规模真实数据集上的两个感知问题，即MegaDepth上的相对相机姿态估计和3DMatch上的点云配准。我们证明SGP达到了与使用地面真相标签训练的受监督甲骨文相同或更高的最新性能。我们将SGP应用于大规模真实数据集上的两个感知问题，即MegaDepth上的相对相机姿态估计和3DMatch上的点云配准。我们证明SGP达到了与使用地面真相标签训练的受监督甲骨文相同或更高的最新性能。我们将SGP应用于大规模真实数据集上的两个感知问题，即MegaDepth上的相对相机姿态估计和3DMatch上的点云配准。我们证明SGP达到了与使用地面真相标签训练的受监督甲骨文相同或更高的最新性能。