Recent efforts in semantic segmentation using deep learning framework have made notable advances. While achieving high performance, however, they often require heavy computation, making them impractical to be used in real world applications. There are two reasons that produce prohibitive computational cost: 1) heavy backbone CNN to create high resolution of contextual information and 2) complex modules to aggregate multi-level features. To address these issues, we propose the computationally efficient architecture called “Sketch-and-Fill Network (SFNet)” with a three-stage Coarse-to-Fine Aggregation (CFA) module for semantic segmentation. In the proposed network, lower-resolution contextual information is first produced so that the overall computation in the backbone CNN is largely reduced. Then, to alleviate the detail loss of the lower-resolution contextual information, the CFA module forms global structures and fills fine details in a coarse-to-fine manner. To preserve global structures, the contextual information is passed without any reduction to the CFA module. Experimental results show that the proposed SFNet achieves significantly lower computational loads while delivering comparable or improved segmentation performance with state-of-the-art methods. Qualitative results show that our method is superior to state-of-the-art methods in capturing fine detail while keeping global structures on Cityscapes, ADE20K and RUGD benchmarks.

中文翻译：

---

用于语义分割的草图填充网络


最近使用深度学习框架进行语义分割的努力取得了显着的进展。然而，在实现高性能的同时，它们通常需要大量计算，这使得它们在现实世界的应用中不切实际。产生过高计算成本的原因有两个：1）用于创建高分辨率上下文信息的重型骨干 CNN；2）用于聚合多级特征的复杂模块。为了解决这些问题，我们提出了一种名为“草图填充网络（SFNet）”的计算高效架构，具有用于语义分割的三阶段粗到细聚合（CFA）模块。在所提出的网络中，首先生成较低分辨率的上下文信息，从而大大减少主干 CNN 中的整体计算量。然后，为了减轻低分辨率上下文信息的细节损失，CFA模块形成全局结构并以从粗到细的方式填充精细细节。为了保留全局结构，上下文信息在不进行任何简化的情况下传递到 CFA 模块。实验结果表明，所提出的 SFNet 显着降低了计算负载，同时使用最先进的方法提供了可比较或改进的分割性能。定性结果表明，我们的方法在捕捉精细细节方面优于最先进的方法，同时保持 Cityscapes、ADE20K 和 RUGD 基准上的全局结构。