Skip to main content
SpringerLink
Log in

A contour self-compensated network for salient object detection

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Given that existing salient object detection methods cannot effectively predict the fine contours of salient objects when extracting local or global contexts and features, we propose a novel contour self-compensated network (CSCNet) to generate a more accurate saliency map with complete contour. Unlike the common binary saliency detection, we reconstruct the salient object detection problem into a multi-classification problem of the background, the salient object, and the salient contour, where the salient contour is used as the third label for ground truth. Meanwhile, the image and its superpixel map are concatenated as the input of our network to add more edge information. Also, a penalty loss is proposed to restrict the spatial relationship between the background, objects, and their contours. Experimentally, we evaluate the proposed CSCNet on six benchmark datasets in both accuracy and efficiency and evaluate the attribute-based performance on the SOC dataset. Compared with 13 state-of-the-art algorithms, our CSCNet can detect salient objects more accurately and completely without adding too many convolutional layers and parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)

    Article  Google Scholar 

  2. Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1597–1604. IEEE (2009)

  3. Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 898–916 (2011)

    Article  Google Scholar 

  4. Borji, A., Cheng, M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24(12), 5706–5722 (2015)

    Article  MathSciNet  Google Scholar 

  5. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)

    Article  Google Scholar 

  6. Chen, S., Tan, X., Wang, B., Hu, X.: Reverse attention for salient object detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 234–250 (2018)

  7. Deng, Z., Hu, X., Zhu, L., Xu, X., Qin, J., Han, G., Heng, P.-A.: R3net: Recurrent residual refinement network for saliency detection. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp. 684–690. AAAI Press (2018)

  8. Fan, D.-P., Cheng, M.-M., Liu, J.-J., Gao, S.-H., Hou, Q., Borji, A.: Salient objects in clutter: bringing salient object detection to the foreground. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 186–202 (2018)

  9. Fan, D.-P., Cheng, M.-M., Liu, Y., Li, T., Borji, A.: Structure-measure: a new way to evaluate foreground maps. In: IEEE International Conference on Computer Vision (ICCV), pp. 4548–4557. IEEE (2017)

  10. Fan, D.-P., Wang, W., Cheng, M.-M., Shen, J.: Shifting more attention to video salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8554–8564 (2019)

  11. Feng, M., Lu, H., Ding, E.: Attentive feedback network for boundary-aware salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1623–1632 (2019)

  12. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  13. Hou, Q., Cheng, M.-M., Hu, X., Borji, A., Tu, Z., Torr, P.H.S.: Deeply supervised salient object detection with short connections, pp. 3203–3212 (2017)

  14. Lee, H., Kim, D.: Salient region-based online object tracking. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1170–1177, March (2018)

  15. Li, G., Xie, Y., Lin, L., Yu, Y.: Instance-level salient object segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 247–256. IEEE (2017)

  16. Li, G., Yu, Y.: Visual saliency based on multiscale deep features. In: Proceedings of the IEEE Conference On Computer Vision and Pattern Recognition, pp. 5455–5463 (2015)

  17. Li, G., Yu, Y.: Deep contrast learning for salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 478–487 (2016)

  18. Li, R., Cai, J., Zhang, H., Wang, T.: Aggregating complementary boundary contrast with smoothing for salient region detection. Vis. Comput. 2017, 1155–1167 (2017)

    Article  Google Scholar 

  19. Li, X., Yang, F., Cheng, H., Liu, W., Shen, D.: Contour knowledge transfer for salient object detection. In: ECCV (2018)

  20. Li, Y., Hou, X., Koch, C., Rehg, J.M., Yuille, A.L.: The secrets of salient object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 280–287 (2014)

  21. Liu, J.-J., Hou, Q., Cheng, M.-M., Feng, J., Jiang, J.: A simple pooling-based design for real-time salient object detection (2019)

  22. Liu, N., Han, J., Yang, M.-H.: Picanet: Learning pixel-wise contextual attention for saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3089–3098 (2018)

  23. Liu, Z., Duan, Q., Shi, S., Zhao, P.: Multi-level progressive parallel attention guided salient object detection for rgb-d images. Vis. Comput. 2020, 1–12 (2020)

    Google Scholar 

  24. Liu, Z., Tang, J., Zhao, P.: Salient object detection via hybrid upsampling and hybrid loss computing. Vis. Comput. 2019D, 1–11 (2019)

    Google Scholar 

  25. Ye, L., Zhou, K., Xiyin, W., Gong, P.: A novel multi-graph framework for salient object detection. Vis. Comput. 35(11), 1683–1699 (2019)

    Article  Google Scholar 

  26. Luo, Z., Mishra, A., Achkar, A., Eichel, J., Li, S., Jodoin, P.-M.: Non-local deep features for salient object detection. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp. 6609–6617 (2017)

  27. Mechrez, R., Shechtman, E., Zelnik-Manor, L.: Saliency driven image manipulation. (2016). CoRR, arXiv:1612.02184

  28. Movahedi, V., Elder, J.H.: Design and perceptual validation of performance measures for salient object segmentation. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, pp. 49–56. IEEE (2010)

  29. Niu, D., Guo, H., Zhao, X., Zhang, C.: Three-dimensional salient point detection based on the laplace-beltrami eigenfunctions. Vis. Comput. 2019, 1–18 (2019)

    Google Scholar 

  30. Perazzi, F., Krähenbühl, P., Pritch, Y., Hornung, A.: Saliency filters: contrast based filtering for salient region detection. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 733–740. IEEE (2012)

  31. Qin, X., He, S., Quintero, C.P., Singh, A., Dehghan, M., Jagersand, M.: Real-time salient closed boundary tracking via line segments perceptual grouping. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4284–4289. IEEE (2017)

  32. Qin, X., He, S., Yang, X., Dehghan, M., Qin, Q., Jägersand, M.: Accurate outline extraction of individual building from very high-resolution optical images. IEEE Geosci. Remote Sens. Lett. 15(11), 1775–1779 (2018)

    Article  Google Scholar 

  33. Qin, X., Zhang, Z., Huang, C., Gao, C., Dehghan, M., Jagersand, M.: Basnet: Boundary-aware salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7479–7489 (2019)

  34. Shen, J., Yunfan, D., Wang, W., Li, X.: Lazy random walks for superpixel segmentation. IEEE Trans. Image Process. 23(4), 1451–1462 (2014)

    Article  MathSciNet  Google Scholar 

  35. Shen, J., Hao, X., Zhiyuan Liang, Y., Liu, W.W., Shao, L.: Real-time superpixel segmentation by dbscan clustering algorithm. IEEE Trans. Image Process. 25(12), 5933–5942 (2016)

    Article  MathSciNet  Google Scholar 

  36. Singh, V.K., Kumar, N.: Saliency bagging: a novel framework for robust salient object detection. Vis. Comput. 2019, 1–19 (2019)

    Google Scholar 

  37. Srivatsa, R.S., Babu, R.V.: Salient object detection via objectness measure. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4481–4485, Sep. (2015)

  38. Sun, J., Ling, H.: Scale and object aware image thumbnailing. Int. J. Comput. Vis. 104(2), 135–153 (2013)

    Article  MathSciNet  Google Scholar 

  39. Tang, Y., Tong, R., Tang, M., Zhang, Y.: Depth incorporating with color improves salient object detection. Vis. Comput. 2016, 111–121 (2016)

    Article  Google Scholar 

  40. Wang, C., Zha, Z.-J., Liu, D., Xie, H.: Robust deep co-saliency detection with group semantic. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8917–8924 (2019)

  41. Wang, L., Lu, H., Ruan, X., Yang, M.: Deep networks for saliency detection via local estimation and global search. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3183–3192, June (2015)

  42. Wang, L., Lu, H., Wang, Y., Feng, M., Wang, D., Yin, B., Ruan, X.: Learning to detect salient objects with image-level supervision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 136–145 (2017)

  43. Wang, L., Wang, L., Lu, H., Zhang, P., Ruan, X.: Saliency detection with recurrent fully convolutional networks. In: European Conference on Computer Vision, pp. 825–841. Springer (2016)

  44. Wang, T., Borji, A., Zhang, L., Zhang, P., Lu, H.: A stagewise refinement model for detecting salient objects in images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4019–4028 (2017)

  45. Wang, W., Lai, Q., Fu, H., Shen, J., Ling, H.: Salient object detection in the deep learning era: an in-depth survey. (2019). arXiv preprint arXiv:1904.09146

  46. Wang, W., Shen, J.: Deep visual attention prediction. IEEE Trans. Image Process. 27(5), 2368–2378 (2017)

    Article  MathSciNet  Google Scholar 

  47. Wang, W., Shen, J., Dong, X., Borji, A.: Salient object detection driven by fixation prediction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1711–1720 (2018)

  48. Wang, W., Shen, J., Dong, X., Borji, A., Yang, R.: Inferring salient objects from human fixations. IEEE Trans. Pattern Anal. Mach. Intell. (2019)

  49. Wang, W., Shen, J., Ling, H.: A deep network solution for attention and aesthetics aware photo cropping. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1531–1544 (2018)

    Article  Google Scholar 

  50. Wang, W., Shen, J., Shao, L.: Video salient object detection via fully convolutional networks. IEEE Trans. Image Process. 27(1), 38–49 (2017)

    Article  MathSciNet  Google Scholar 

  51. Wang, W., Shen, J., Yang, R., Porikli, F.: Saliency-aware video object segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 40(1), 20–33 (2017)

    Article  Google Scholar 

  52. Wang, W., Shen, J., Yizhou, Y., Ma, K.-L.: Stereoscopic thumbnail creation via efficient stereo saliency detection. IEEE Trans. Vis. Comput. Graph. 23(8), 2014–2027 (2016)

    Article  Google Scholar 

  53. Wang, W., Zhao, S., Shen, J., Hoi, S., Borji, A.: Salient object detection with pyramid attention and salient edges. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 04 (2019)

  54. Wang, X., Ma, H., Chen, X.: Salient object detection via fast r-cnn and low-level cues. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 1042–1046, Sep. (2016)

  55. Wu, R., Feng, M., Guan, W., Wang, D., Lu, H., Ding, E.: A mutual learning method for salient object detection with intertwined multi-supervision. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

  56. Wu, Z., Su, L., Huang, Q.: Stacked cross refinement network for edge-aware salient object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 7264–7273 (2019)

  57. Yan, Q., Xu, L., Shi, J., Jia, J.: Hierarchical saliency detection. In: Proceedings of the IEEE Conference On Computer Vision And Pattern Recognition (CVPR), pp. 1155–1162 (2013)

  58. Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.-H.: Saliency detection via graph-based manifold ranking. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, June 23–28, 2013, pp. 3166–3173 (2013)

  59. Zhang, F., Du, B., Zhang, L.: Saliency-guided unsupervised feature learning for scene classification. IEEE Trans. Geosci. Remote Sens. 53(4), 2175–2184 (2014)

    Article  Google Scholar 

  60. Zhang, J., Dai, Y., Porikli, F., He, M.: Deep edge-aware saliency detection. (2017). CoRR, arXiv:1708.04366

  61. Zhang, L., Dai, J., Lu, H., He, Y., Wang, G.: A bi-directional message passing model for salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1741–1750 (2018)

  62. Zhang, P., Wang, D., Lu, H., Wang, H., Ruan, X.: Amulet: Aggregating multi-level convolutional features for salient object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 202–211 (2017)

  63. Zhang, P., Wang, D., Lu, H., Wang, H., Yin, B.: Learning uncertain convolutional features for accurate saliency detection. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 212–221. IEEE (2017)

  64. Zhao, J.-X., Cao, Y., Fan, D.-P., Cheng, M.-M., Li, X.-Y., Zhang, L.: Contrast prior and fluid pyramid integration for rgbd salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3927–3936 (2019)

  65. Zhao, J.-X., Liu, J.-J., Fan, D.-P., Cao, Y., Yang, J., Cheng, M.-M.: Egnet: Edge guidance network for salient object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 8779–8788 (2019)

  66. Zhao, J., Bo, R., Hou, Q., Cheng, M.-M., Rosin, P.: Flic: fast linear iterative clustering with active search. Comput. Vis. Med. 4(4), 333–348 (2018)

    Article  Google Scholar 

  67. Zhao, R., Ouyang, W., Li, H., Wang, X.: Saliency detection by multi-context deep learning. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1265–1274, June (2015)

  68. Zhu, W., Liang, S., Wei, Y., Sun, J.: Saliency optimization from robust background detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2814–2821, 09 (2014)

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No. 51905529), Shaanxi Province Natural Science-Based Research Program (2019JQ-295), and the UCAS PhD scholarship. Thanks to teacher Adams at NTU for providing equipment and to Yew Lee Tan for proofreading and modifying this paper.

Author information

Authors and Affiliations

  1. Xi’an Institute of Optics and Precision Mechanics of CAS, Xi’an, China

    Yanan Wang, Huawei Wang & Jianzhong Cao

  2. University of Chinese Academy of Sciences, Beijing, China

    Yanan Wang

Authors
  1. Yanan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huawei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianzhong Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanan Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Wang, H. & Cao, J. A contour self-compensated network for salient object detection. Vis Comput 37, 1467–1479 (2021). https://doi.org/10.1007/s00371-020-01882-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01882-w

Keywords

Search

Navigation