Abstract Many convolutional neural network (CNN)-based approaches for stereoscopic salient object detection involve fusing either low-level or high-level features from the color and disparity channels. The former method generally produces incomplete objects, whereas the latter tends to blur object boundaries. In this paper, a coupled CNN (CoCNN) is proposed to fuse color and disparity features from low to high layers in a unified deep model. It consists of three parts: two parallel multilinear span networks, a cascaded span network and a conditional random field module. We first apply the multilinear span network to compute multiscale saliency predictions based on RGB and disparity individually. Each prediction, learned under separate supervision, utilizes the multilevel features extracted by the multilinear span network. Second, a proposed cascaded span network, under deep supervision, is designed as a coupling unit to fuse the two feature streams at each scale and integrate all fused features in a supervised manner to construct a saliency map. Finally, we formulate a constraint in the form of a conditional random field model to refine the saliency map based on the a priori assumption that objects with similar saliency values have similar colors and disparities. Experiments conducted on two commonly used datasets demonstrate that the proposed method outperforms previous state-of-the-art methods.

中文翻译：

---

CoCNN：用于立体显着目标检测的 RGB-D 深度融合

摘要 许多基于卷积神经网络 (CNN) 的立体显着对象检测方法涉及融合来自颜色和视差通道的低级或高级特征。前一种方法通常会产生不完整的对象，而后一种方法往往会模糊对象边界。在本文中，提出了一种耦合 CNN（CoCNN）来在统一的深度模型中融合从低层到高层的颜色和视差特征。它由三部分组成：两个平行的多线性跨度网络、一个级联的跨度网络和一个条件随机场模块。我们首先应用多线性跨度网络来分别计算基于 RGB 和视差的多尺度显着性预测。每个预测都是在单独的监督下学习的，利用多线性跨度网络提取的多级特征。第二，在深度监督下，建议的级联跨度网络被设计为耦合单元，以融合每个尺度的两个特征流，并以受监督的方式整合所有融合特征以构建显着图。最后，我们以条件随机场模型的形式制定约束，以基于具有相似显着值的对象具有相似颜色和视差的先验假设来细化显着图。在两个常用数据集上进行的实验表明，所提出的方法优于以前最先进的方法。我们以条件随机场模型的形式制定了一个约束，以基于具有相似显着值的对象具有相似的颜色和视差的先验假设来细化显着图。在两个常用数据集上进行的实验表明，所提出的方法优于以前最先进的方法。我们以条件随机场模型的形式制定约束，以基于具有相似显着值的对象具有相似颜色和视差的先验假设来细化显着图。在两个常用数据集上进行的实验表明，所提出的方法优于以前最先进的方法。