Dense image alignment from RGB-D images remains a critical issue for real-world applications, especially under challenging lighting conditions and in a wide baseline setting. In this letter, we propose a new framework to learn a pixel-wise deep feature map and a deep feature-metric uncertainty map predicted by a Convolutional Neural Network (CNN), which together formulate a deep probabilistic feature-metric residual of the two-view constraint that can be minimised using Gauss-Newton in a coarse-to-fine optimisation framework. Furthermore, our network predicts a deep initial pose for faster and more reliable convergence. The optimisation steps are differentiable and unrolled to train in an end-to-end fashion. Due to its probabilistic essence, our approach can easily couple with other residuals, where we show a combination with ICP. Experimental results demonstrate state-of-the-art performances on the TUM RGB-D dataset and the 3D rigid object tracking dataset. We further demonstrate our method's robustness and convergence qualitatively.

中文翻译：

---

深度概率特征度量跟踪

RGB-D 图像的密集图像对齐仍然是实际应用的关键问题，尤其是在具有挑战性的照明条件和宽基线设置下。在这封信中，我们提出了一个新框架来学习由卷积神经网络 (CNN) 预测的像素级深度特征图和深度特征度量不确定性图，它们共同制定了两个深度概率特征度量残差：可以在粗到细优化框架中使用 Gauss-Newton 最小化视图约束。此外，我们的网络预测了深度初始姿势，以实现更快、更可靠的收敛。优化步骤是可微分和展开的，以端到端的方式进行训练。由于其概率本质，我们的方法可以很容易地与其他残差结合，在那里我们展示了与 ICP 的组合。实验结果证明了在 TUM RGB-D 数据集和 3D 刚性对象跟踪数据集上的最新性能。我们进一步定性地证明了我们方法的鲁棒性和收敛性。