Monocular depth estimation by unsupervised learning is a potential strategy, which is mainly self-supervised by calculating view reconstruction loss from stereo pairs or monocular sequences. However, most existing works only consider the geometric information during training, without using semantics. We propose a semantic monocular depth estimation (SE-Net), a neural network framework that estimates depth using semantic information and video sequences. The whole framework is semi-supervised, because we take advantage of labelled semantic ground truth data. In view of the structural consistency between the semantically segmented image and the depth map, we first perform semantic segmentation on the image, and then use the semantic labels to guide the construction of the depth estimation network. Experiments on the KITTI dataset show that learning semantic information from images can effectively improve the effect of monocular depth estimation, and SE-Net is superior to the most advanced methods in depth estimation accuracy.

通过无监督学习进行单眼深度估计是一种潜在的策略，它主要通过计算立体对或单眼序列的视图重建损失进行自我监督。但是，大多数现有作品仅在训练过程中考虑几何信息，而没有使用语义。我们提出了语义单眼深度估计（SE-Net），这是一种使用语义信息和视频序列估计深度的神经网络框架。整个框架是半监督的，因为我们利用了标记的语义基础事实数据。考虑到语义分割图像和深度图之间的结构一致性，我们首先对图像进行语义分割，然后使用语义标签指导深度估计网络的构建。