Exploiting both RGB (2D appearance) and Depth (3D geometry) information can improve the performance of semantic segmentation. However, due to the inherent difference between the RGB and Depth information, it remains a challenging problem in how to integrate RGB-D features effectively. In this letter, to address this issue, we propose a Non-local Aggregation Network (NANet), with a well-designed Multi-modality Non-local Aggregation Module (MNAM), to better exploit the non-local context of RGB-D features at multi-stage. Compared with most existing RGB-D semantic segmentation schemes, which only exploit local RGB-D features, the MNAM enables the aggregation of non-local RGB-D information along both spatial and channel dimensions. The proposed NANet achieves comparable performances with state-of-the-art methods on popular RGB-D benchmarks, NYUDv2 and SUN-RGBD.

中文翻译：

---

RGB-D语义分割的非局部聚合

同时利用RGB（2D外观）和Depth（3D几何）信息可以提高语义分割的性能。但是，由于RGB和深度信息之间的固有差异，如何有效地集成RGB-D功能仍然是一个具有挑战性的问题。在这封信中，为了解决这个问题，我们提出了一个具有精心设计的多模式非本地聚合模块（MNAM）的非本地聚合网络（NANet），以更好地利用RGB-D的非本地上下文多阶段功能。与仅利用本地RGB-D功能的大多数现有RGB-D语义分割方案相比，MNAM可以沿空间和通道维度聚合非本地RGB-D信息。拟议的NANet使用流行的RGB-D基准测试中的最新方法可达到可比的性能，