Depth estimation from stereo images is carried out with unmatched results by convolutional neural networks trained end-to-end to regress dense disparities. Like for most tasks, this is possible if large amounts of labelled samples are available for training, possibly covering the whole data distribution encountered at deployment time. Being such an assumption systematically unmet in real applications, the capacity of adapting to any unseen setting becomes of paramount importance. Purposely, we propose a continual adaptation paradigm for deep stereo networks designed to deal with challenging and ever-changing environments. We design a lightweight and modular architecture, Modularly ADaptive Network (MADNet), and formulate Modular ADaptation algorithms (MAD, MAD++) which permit efficient optimization of independent sub-portions of the entire network. In our paradigm, the learning signals needed to continuously adapt models online can be sourced from self-supervision via right-to-left image warping or from traditional stereo algorithms. With both sources, no other data than the input images being gathered at deployment time are needed. Thus, our network architecture and adaptation algorithms realize the first real-time self-adaptive deep stereo system and pave the way for a new paradigm that can facilitate practical deployment of end-to-end architectures for dense disparity regression.

中文翻译：

---

持续适应深度立体声


通过端到端训练的卷积神经网络进行立体图像的深度估计，以回归密集视差，结果无与伦比。与大多数任务一样，如果有大量标记样本可用于训练，并且可能覆盖部署时遇到的整个数据分布，这是可能的。由于这种假设在实际应用中系统性地未得到满足，因此适应任何看不见的环境的能力变得至关重要。我们有目的地为深度立体网络提出了一种持续适应范式，旨在应对具有挑战性和不断变化的环境。我们设计了一种轻量级模块化架构，即模块化自适应网络（MADNet），并制定了模块化自适应算法（MAD、MAD++），允许对整个网络的独立子部分进行有效优化。在我们的范例中，持续在线调整模型所需的学习信号可以来自通过从右到左图像扭曲的自我监督或传统的立体算法。对于这两个源，除了在部署时收集的输入图像之外不需要其他数据。因此，我们的网络架构和自适应算法实现了第一个实时自适应深度立体系统，并为新的范式铺平了道路，该范式可以促进密集视差回归的端到端架构的实际部署。