We propose an approach for exploiting contextual information in semantic image segmentation, and particularly investigate the use of patch-patch context and patch-background context in deep CNNs. We formulate deep structured models by combining CNNs and Conditional Random Fields (CRFs) for learning the patch-patch context between image regions. Specifically, we formulate CNN-based pairwise potential functions to capture semantic correlations between neighboring patches. Efficient piecewise training of the proposed deep structured model is then applied in order to avoid repeated expensive CRF inference during the course of back propagation. For capturing the patch-background context, we show that a network design with traditional multi-scale image inputs and sliding pyramid pooling is very effective for improving performance. We perform comprehensive evaluation of the proposed method. We achieve new state-of-the-art performance on a number of challenging semantic segmentation datasets.

中文翻译：

---

使用深层结构化模型探索上下文以进行语义分割


我们提出了一种在语义图像分割中利用上下文信息的方法，并特别研究了深度 CNN 中补丁-补丁上下文和补丁-背景上下文的使用。我们通过结合 CNN 和条件随机场 (CRF) 来制定深度结构化模型，用于学习图像区域之间的补丁上下文。具体来说，我们制定基于 CNN 的成对势函数来捕获相邻补丁之间的语义相关性。然后对所提出的深度结构化模型进行有效的分段训练，以避免在反向传播过程中重复进行昂贵的 CRF 推理。为了捕获补丁背景上下文，我们表明使用传统多尺度图像输入和滑动金字塔池的网络设计对于提高性能非常有效。我们对所提出的方法进行综合评估。我们在许多具有挑战性的语义分割数据集上实现了最先进的性能。