Visual saliency prediction for RGB-D images is more challenging than that for their RGB counterparts. Additionally, very few investigations have been undertaken concerning RGB-D-saliency prediction. The proposed study presents a method based on a hierarchical multimodal adaptive fusion (HMAF) network to facilitate end-to-end prediction of RGB-D saliency. In the proposed method, hierarchical (multilevel) multimodal features are first extracted from an RGB image and depth map using a VGG-16-based two-stream network. Subsequently, the most significant hierarchical features of the said RGB image and depth map are predicted using three two-input attention modules. Furthermore, adaptive fusion of saliencies concerning the above-mentioned fused saliency features of different levels (hierarchical fusion saliency features) can be accomplished using a three-input attention module to facilitate high-accuracy RGB-D visual saliency prediction. Comparisons based on the application of the proposed HMAF-based approach against those of other state-of-the-art techniques on two challenging RGB-D datasets demonstrate that the proposed method outperforms other competing approaches consistently by a considerable margin.

中文翻译：

---

用于预测RGB-D显着性的分层多模态自适应融合（HMAF）网络

RGB-D图像的视觉显着性预测比RGB对应图像的显着性预测更具挑战性。另外，关于RGB-D显着性预测的研究很少。提出的研究提出了一种基于分层多模态自适应融合（HMAF）网络的方法，以促进RGB-D显着性的端到端预测。在提出的方法中，首先使用基于VGG-16的两流网络从RGB图像和深度图中提取分层（多级）多峰特征。随后，使用三个二输入注意力模块来预测所述RGB图像和深度图的最重要的分层特征。此外，可以使用三输入注意模块来促进高精度的RGB-D视觉显着性预测，以实现涉及上述不同级别的融合显着性特征的显着性自适应融合（分层融合显着性特征）。在两个具有挑战性的RGB-D数据集上，基于所提出的基于HMAF的方法与其他最新技术的应用的比较表明，所提出的方法始终在性能上优于其他竞争方法。