Stereo matching aims to estimating disparity by finding the correspondence of each pixel between two images which is crucial to 3D scene reconstruction. Nowadays 3D convolution neural networks achieve impressive performances on stereo matching. However, it is memory consuming and computation complex. And it is challenging to finding the corresponding pixels in textureless and near boundary regions. Therefore, a stereo matching neural network is proposed which use semantic segmentation and boundary detection task to improve the accuracy of stereo matching near boundary and textureless regions. And a hybrid cost volume which reflects the similarity between left and right feature map, is designed to contains semantic cost volume and boundary cost volume with attention mechanism. The stereo matching neural network is designed to rely on coarse-to-fine strategy which predict a complete disparity map at the highest resolution and refine disparity at the lower resolution. We conduct comprehensive experiments on KITTI 2015 datasets, and compare with some recent stereo matching neural networks, the D1-all (3-pixel error) is 2.8% and run time is 0.044s which shows that embedding semantic and boundary information can improve the accuracy of stereo matching.

立体匹配旨在通过找到两个图像之间每个像素的对应关系来估计视差，这对 3D 场景重建至关重要。如今，3D 卷积神经网络在立体匹配方面取得了令人印象深刻的表现。但是，它占用内存且计算复杂。并且在无纹理和近边界区域中找到相应的像素具有挑战性。因此，提出了一种立体匹配神经网络，它使用语义分割和边界检测任务来提高近边界和无纹理区域的立体匹配精度。并设计了一个反映左右特征图相似性的混合成本量，通过注意机制包含语义成本量和边界成本量。立体匹配神经网络的设计依赖于从粗到细的策略，该策略以最高分辨率预测完整的视差图，并在较低分辨率下细化视差。我们在 KITTI 2015 数据集上进行了全面的实验，并与最近的一些立体匹配神经网络进行了比较，D1-all（3 像素误差）为 2.8%，运行时间为 0.044 秒，这表明嵌入语义和边界信息可以提高准确性立体匹配。