In this paper, we propose a novel end-to-end deep neural network model for omnidirectional depth estimation from a wide-baseline multi-view stereo setup. The images captured with ultra-wide field-of-view cameras on an omnidirectional rig are processed by the feature extraction module, and then the deep feature maps are warped onto the concentric spheres swept through all candidate depths using the calibrated camera parameters. The 3D encoder-decoder block takes the aligned feature volume to produce an omnidirectional depth estimate with regularization on uncertain regions utilizing the global context information. For more accurate depth estimation we also propose an uncertainty prior guidance in two ways: depth map filtering and guiding regularization. In addition, we present large-scale synthetic datasets for training and testing omnidirectional multi-view stereo algorithms. Our datasets consist of 13K ground-truth depth maps and 53K fisheye images in four orthogonal directions with various objects and environments. Experimental results show that the proposed method generates excellent results in both synthetic and real-world environments, and it outperforms the prior art and the omnidirectional versions of the state-of-the-art conventional stereo algorithms.

中文翻译：

---

具有不确定性先验的全向立体匹配的端到端学习


在本文中，我们提出了一种新颖的端到端深度神经网络模型，用于从宽基线多视图立体设置进行全向深度估计。全向装置上超宽视场相机捕获的图像由特征提取模块处理，然后使用校准的相机参数将深度特征图扭曲到扫过所有候选深度的同心球上。 3D 编码器-解码器块采用对齐的特征量来生成全向深度估计，并利用全局上下文信息对不确定区域进行正则化。为了更准确的深度估计，我们还通过两种方式提出了不确定性先验指导：深度图过滤和指导正则化。此外，我们还提供了用于训练和测试全向多视图立体算法的大规模合成数据集。我们的数据集由 13K 地面真实深度图和 53K 鱼眼图像组成，在四个正交方向上具有各种物体和环境。实验结果表明，所提出的方法在合成和现实环境中都产生了出色的结果，并且优于现有技术和最先进的传统立体算法的全向版本。