Semantic segmentation is a fundamental task in computer vision, and it has various applications in fields such as robotic sensing, video surveillance, and autonomous driving. A major research topic in urban road semantic segmentation is the proper integration and use of cross-modal information for fusion. Here, we attempt to leverage inherent multimodal information and acquire graded features to develop a novel multilabel-learning network for RGB-thermal urban scene semantic segmentation. Specifically, we propose a strategy for graded-feature extraction to split multilevel features into junior, intermediate, and senior levels. Then, we integrate RGB and thermal modalities with two distinct fusion modules, namely a shallow feature fusion module and deep feature fusion module for junior and senior features. Finally, we use multilabel supervision to optimize the network in terms of semantic, binary, and boundary characteristics. Experimental results confirm that the proposed architecture, the graded-feature multilabel-learning network, outperforms state-of-the-art methods for urban scene semantic segmentation, and it can be generalized to depth data.

中文翻译：

---

GMNet：用于 RGB 热城市场景语义分割的分级特征多标签学习网络


语义分割是计算机视觉中的一项基本任务，在机器人传感、视频监控和自动驾驶等领域有着广泛的应用。城市道路语义分割的一个主要研究课题是跨模态信息的适当整合和利用以进行融合。在这里，我们尝试利用固有的多模态信息并获取分级特征来开发一种新颖的多标签学习网络，用于 RGB 热城市场景语义分割。具体来说，我们提出了一种分级特征提取策略，将多级特征分为初级、中级和高级。然后，我们将 RGB 和热模态与两个不同的融合模块集成，即浅层特征融合模块和针对初级和高级特征的深层特征融合模块。最后，我们使用多标签监督在语义、二进制和边界特征方面优化网络。实验结果证实，所提出的体系结构（分级特征多标签学习网络）优于城市场景语义分割的最先进方法，并且可以推广到深度数据。