Abstract
The topological perception theory claims that visual perception of a scene begins from topological properties and then exploits local details. Inspired by this theory, we defined the topological descriptor and topological complexity, and we observed, based on statistics, that the saliencies of the regions with higher topological complexities are generally higher than those of regions with lower topological complexities. We then introduced the topological complexity as a saliency prior and proposed a novel unsupervised topo-prior-guided saliency detection system (TOPS). This system is framed as a topological saliency prior (topo-prior)-guided two-level local cue processing (i.e., pixel- and regional-level cues) with a multi-scale strategy, which includes three main modules: (1) a basic computational model of the topological perception theory for extracting topological features from images, (2) a topo-prior calculation method based on the topological features, and (3) a global–local saliency combination framework guided by the topo-prior. Extensive experiments on widely used salient object detection (SOD) datasets demonstrate that our system outperforms the unsupervised state-of-the-art algorithms. In addition, the topo-prior proposed in this work can be used to boost supervised methods including the deep-learning-based ones for fixation prediction and SOD tasks.
Similar content being viewed by others
References
Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency tuned salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1597–1604
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., & Süsstrunk, S. (2012). Slic superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11), 2274–2282.
Adelson, E. H., Anderson, C. H., Bergen, J. R., Burt, P. J., & Ogden, J. M. (1984). Pyramid methods in image processing. RCA engineer, 29(6), 33–41.
Arbelaez P (2006) Boundary extraction in natural images using ultrametric contour maps. In: Conference on IEEE Conference on Computer Vision and Pattern Recognition Workshop, pp 182–182
Borji A (2019) Saliency prediction in the deep learning era: Successes and limitations. IEEE Transactions on Pattern Analysis and Machine Intelligence
Borji A, Itti L (2011) Scene classification with a sparse set of salient regions. IEEE International Conference on Robotics and Automation pp 1902–1908
Borji, A., & Itti, L. (2012). State-of-the-art in visual attention modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 185–207.
Borji, A., Sihite, D. N., & Itti, L. (2012). Quantitative analysis of human-model agreement in visual saliency modeling: A comparative study. IEEE Transactions on Image Processing, 22(1), 55–69.
Borji A, Cheng MM, Hou Q, Jiang H, Li J (2014) Salient object detection: A survey. Computational Visual Media pp 1–34
Bruce, N. D., & Tsotsos, J. K. (2009). Saliency, attention, and visual search: An information theoretic approach. Journal of vision, 9(3), 5.
Chen J, Li Q, Wu W, Ling H, Wu L, Zhang B, Li P (2019) Saliency detection via topological feature modulated deep learning. 2019 IEEE International Conference on Image Processing pp 1630–1634
Chen, L. (1982). Topological structure in visual perception. Science, 218(4573), 699–700.
Chen, L. (2005). The topological approach to perceptual organization. Visual Cognition, 12(4), 553–637.
Chen, L., Zhang, S., & Mandyam, V. S. (2003). Global perception in small brains: topological pattern recognition in honey bees. Proceedings of the National Academy of Sciences of the United States of America, 100(11), 6884–6889.
Chen, S., Zheng, L., Hu, X., & Zhou, P. (2016). Discriminative saliency propagation with sink points. Pattern recognition, 60, 2–12.
Chen S, Tan X, Wang B, Hu X (2018) Reverse attention for salient object detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 234–250
Chen, S., Tan, X., Wang, B., Lu, H., Hu, X., & Fu, Y. (2020). Reverse attention-based residual network for salient object detection. IEEE Transactions on Image Processing, 29, 3763–3776.
Chen X, Zheng A, Li J, Lu F (2017) Look, perceive and segment: Finding the salient objects in images via two-stream fixation-semantic cnns. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1050–1058
Cheng MM, Zhang GX, Mitra NJ, Huang X, Hu SM (2011) Global contrast based salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 409–416
Cheng, M. M., Mitra, N. J., Huang, X., Torr, P. H., & Hu, S. M. (2014a). Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3), 569–582.
Cheng MM, Zhang Z, Lin WY, Torr P (2014b) Bing: Binarized normed gradients for objectness estimation at 300fps. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3286–3293
Cong, R., Lei, J., Fu, H., Cheng, M. M., Lin, W., & Huang, Q. (2019). Review of visual saliency detection with comprehensive information. IEEE Transactions on Circuits and Systems for Video Technology, 29(10), 2941–2959.
Cornia M, Baraldi L, Serra G, Cucchiara R (2016) A deep multi-level network for saliency prediction. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp 3488–3493
DeYoe, E. A., & Van Essen, D. C. (1988). Concurrent processing streams in monkey visual cortex. Trends in neurosciences, 11(5), 219–226.
Fan DP, Cheng MM, Liu Y, Li T, Borji A (2017) Structure-measure: A new way to evaluate foreground maps. In: IEEE International Conference on Computer Vision, pp 4548–4557
Fang, Y., Wang, J., Narwaria, M., Le Callet, P., & Lin, W. (2014). Saliency detection for stereoscopic images. IEEE Transactions on Image Processing, 23(6), 2625–2636.
Gao, Y., Shi, M., Tao, D., & Xu, C. (2015). Database saliency for fast image retrieval. IEEE Transactions on Multimedia, 17(3), 359–369.
Garcia-Diaz, A., Fdez-Vidal, X. R., Pardo, X. M., & Dosil, R. (2012). Saliency from hierarchical adaptation through decorrelation and variance normalization. Image and Vision Computing, 30(1), 51–64.
Gong C, Tao D, Liu W, Maybank SJ, Fang M, Fu K, Yang J (2015) Saliency propagation from simple to difficult. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2531–2539
Gu, X., Fang, Y., & Wang, Y. (2013). Attention selection using global topological properties based on pulse coupled neural network. Computer Vision Image Understanding, 117(10), 1400–1411.
Harel J, Koch C, Perona P (2007) Graph-based visual saliency. In: Advances in Neural Information Processing Systems, pp 545–552
He, L., Zhou, K., Zhou, T., He, S., & Chen, L. (2015). Topology-defined units in numerosity perception. Proceedings of the National Academy of Sciences, 112(41), E5647–E5655.
He S, Tavakoli HR, Borji A, Mi Y, Pugeault N (2019) Understanding and visualizing deep visual saliency models. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 10206–10215
Heeger DJ, Bergen JR (1995) Pyramid-based texture analysis/synthesis. In: the 22nd annual conference on Computer Graphics and Interactive Techniques, Citeseer, pp 229–238
Hornung A, Pritch Y, Krahenbuhl P, Perazzi F (2012) Saliency filters: Contrast based filtering for salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 733–740
Hou X, Zhang L (2007) Saliency detection: A spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1–8
Hou, X., Harel, J., & Koch, C. (2011). Image signature: Highlighting sparse salient regions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(1), 194–201.
Huang, X., & Zhang, Y. (2018). Water flow driven salient object detection at 180 fps. Pattern Recognition, 76, 95–107.
Huang, X., & Zhang, Y. J. (2017). 300-fps salient object detection via minimum directional contrast. IEEE Transactions on Image Processing, 26(9), 4243–4254.
Huang Y, Huang K, Tan T, Tao D (2009) A novel visual organization based on topological perception. In: Asian Conference on Computer Vision, Springer, pp 180–189
Itti, L., & Koch, C. (2001). Computational modelling of visual attention. Nature Reviews Neuroscience, 2(3), 194–203.
Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, 1254–1259.
Ji, Q., Fang, Z., Xie, Z., & Lu, Z. (2013). Video abstraction based on the visual attention model and online clustering. Signal Processing-image Communication, 28(3), 241–253.
Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013a) Salient object detection: A discriminative regional feature integration approach. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2083–2090
Jiang P, Ling H, Yu J, Peng J (2013b) Salient region detection by ufo: Uniqueness, focusness and objectness. In: IEEE International Conference on Computer Vision, pp 1976–1983
Judd T, Ehinger K, Durand F, Torralba A (2009) Learning to predict where humans look. In: IEEE International Conference on Computer Vision, IEEE, pp 2106–2113
Kim, J., Han, D., Tai, Y. W., & Kim, J. (2015). Salient region detection via high-dimensional color transform and local spatial support. IEEE transactions on image processing, 25(1), 9–23.
Klingner M, Termöhlen JA, Mikolajczyk J, Fingscheidt T (2020) Self-supervised monocular depth estimation: Solving the dynamic object problem by semantic guidance. In: European Conference on Computer Vision, Springer, pp 582–600
Koch C, Ullman S (1987) Shifts in selective visual attention: towards the underlying neural circuitry. In: Matters of intelligence, Springer, pp 115–141
Koffka KPrinciples of Gestalt psychologyPrinciples of Gestalt psychology. Routledge
Kruthiventi, S. S., Ayush, K., & Babu, R. V. (2017). Deepfix: A fully convolutional neural network for predicting human eye fixations. IEEE Transactions on Image Processing, 26(9), 4446–4456.
Kummerer M, Wallis TSA, Gatys LA, Bethge M (2017) Understanding low- and high-level contributions to fixation prediction. In: The IEEE International Conference on Computer Vision (ICCV)
Li C, Yuan Y, Cai W, Xia Y, Dagan Feng D (2015a) Robust saliency detection via regularized random walks ranking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2710–2717
Li G, Yu Y (2015) Visual saliency based on multiscale deep features. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 5455–5463
Li G, Xie Y, Wei T, Wang K, Lin L (2018) Flow guided recurrent neural encoder for video salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3243–3252
Li, J., Levine, M. D., An, X., Xu, X., & He, H. (2013). Visual saliency based on scale-space analysis in the frequency domain. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(4), 996–1010.
Li N, Sun B, Yu J (2015b) A weighted sparse coding framework for saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 5216–5223
Lin, X., Wang, Z. J., Ma, L., & Wu, X. (2019). Saliency detection via multi-scale global cues. IEEE Transactions on Multimedia, 21(7), 1646–1659.
Liu N, Han J, Zhang D, Wen S, Liu T (2015) Predicting eye fixations using convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 362–370
Liu N, Han J, Yang MH (2018) Picanet: Learning pixel-wise contextual attention for saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3089–3098
Liu, Q., Hong, X., Zou, B., Chen, J., Chen, Z., & Zhao, G. (2017). Hierarchical contour closure-based holistic salient object detection. IEEE Transactions on Image Processing, 26(9), 4537–4552.
Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., et al. (2010). Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(2), 353–367.
Livingstone, M. S., & Hubel, D. H. (1987). Psychophysical evidence for separate channels for the perception of form, color, movement, and depth. Journal of Neuroscience, 7(11), 3416–3468.
Ma, C., Miao, Z., Zhang, X. P., & Li, M. (2017). A saliency prior context model for real-time object tracking. IEEE Transactions on Multimedia, 19(11), 2415–2424.
Marr, David (1982) Vision: A computational investigation into the human representation and processing of visual information. Quarterly Review of Biology 8
Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1), 62–66.
Peng H, Li B, Xiong W, Hu W, Ji R (2014) Rgbd salient object detection: A benchmark and algorithms. In: European Conference on Computer Vision, Springer, pp 92–109
Peng, H., Li, B., Ling, H., Hu, W., Xiong, W., & Maybank, S. J. (2016). Salient object detection via structured matrix decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 818–832.
Perazzi F, Krähenbühl P, Pritch Y, Hornung A (2012) Saliency filters: Contrast based filtering for salient region detection. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp 733–740
Peters, R. J., Iyer, A., Itti, L., & Koch, C. (2005). Components of bottom-up gaze allocation in natural images. Vision research, 45(18), 2397–2416.
Qin Y, Lu H, Xu Y, Wang H (2015) Saliency detection via cellular automata. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 110–119
Qin, Y., Feng, M., Lu, H., & Cottrell, G. W. (2018). Hierarchical cellular automata for visual saliency. International Journal of Computer Vision, 126(7), 751–770.
Qu, L., He, S., Zhang, J., Tian, J., Tang, Y., & Yang, Q. (2017). Rgbd salient object detection via deep fusion. IEEE Transactions on Image Processing, 26(5), 2274–2285.
Rahtu E, Kannala J, Salo M, Heikkilä J (2010) Segmenting salient objects from images and videos. European Conference on Computer Vision pp 366–379
Scharfenberger C, Wong A, Fergani K, Zelek JS, Clausi DA (2013) Statistical textural distinctiveness for salient region detection in natural images. In: IEEE Conference on Computer Vision and Pattern Recognition
Seki A, Pollefeys M (2017) Sgm-nets: Semi-global matching with neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 231–240
Shi, J., Yan, Q., Xu, L., & Jia, J. (2016). Hierarchical image saliency detection on extended cssd. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(4), 717–729.
Siva P, Russell C, Xiang T, Agapito L (2013) Looking beyond the image: Unsupervised learning for object saliency and detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3238–3245
Song, H., Liu, Z., Du, H., Sun, G., Le Meur, O., & Ren, T. (2017). Depth-aware salient object detection and segmentation via multiscale discriminative saliency fusion and bootstrap learning. IEEE Transactions on Image Processing, 26(9), 4204–4216.
Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12(1), 97–136.
Tu WC, He S, Yang Q, Chien SY (2016) Real-time salient object detection with a minimum spanning tree. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2334–2342
Vig E, Dorr M, Cox D (2014) Large-scale optimization of hierarchical features for saliency prediction in natural images. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2798–2805
Wang, B., Zhou, T. G., Zhuo, Y., & Chen, L. (2007). Global topological dominance in the left hemisphere. Proceedings of the National Academy of Sciences, 104(52), 21014–21019.
Wang, J., Jiang, H., Yuan, Z., Cheng, M. M., Hu, X., & Zheng, N. (2017a). Salient object detection: A discriminative regional feature integration approach. International Journal of Computer Vision, 123(2), 251–268.
Wang L, Wang L, Lu H, Zhang P, Ruan X (2016) Saliency detection with recurrent fully convolutional networks. In: European Conference on Computer Vision, Springer, pp 825–841
Wang L, Lu H, Wang Y, Feng M, Wang D, Yin B, Ruan X (2017b) Learning to detect salient objects with image-level supervision. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 136–145
Wang T, Zhang L, Wang S, Lu H, Yang G, Ruan X, Borji A (2018) Detect globally, refine locally: A novel approach to saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3127–3135
Wang, W., & Shen, J. (2018). Deep visual attention prediction. IEEE Transactions on Image Processing, 27(5), 2368–2378.
Wang W, Lai Q, Fu H, Shen J, Ling H (2019a) Salient object detection in the deep learning era: An in-depth survey. arXiv preprint arXiv:1904.09146
Wang W, Shen J, Cheng MM, Shao L (2019b) An iterative and cooperative top-down and bottom-up inference network for salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 5968–5977
Wang W, Shen J, Dong X, Borji A, Yang R (2019c) Inferring salient objects from human fixations. IEEE Transactions on Pattern Analysis and Machine Intelligence pp 1
Wang W, Shen J, Xie J, Cheng MM, Ling H, Borji A (2019d) Revisiting video saliency prediction in the deep learning era. IEEE Transactions on Pattern Analysis and Machine Intelligence pp 1
Wang W, Zhao S, Shen J, Hoi SC, Borji A (2019e) Salient object detection with pyramid attention and salient edges. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1448–1457
Wei Y, Wen F, Zhu W, Sun J (2012) Geodesic saliency using background priors. In: European Conference on Computer Vision, Springer, pp 29–42
Wolfe JM (1994) Guided search 2.0 a revised model of visual search. Psychon Bull Rev 1(2):202–238
Wolfe, J. M., Melissa, L.-H. V., Evans, K. K., & Greene, M. R. (2011). Visual search in scenes involves selective and nonselective pathways. Trends in Cognitive Sciences, 15(2), 77–84.
Wu Z, Su L, Huang Q (2019) Cascaded partial decoder for fast and accurate salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3907–3916
Xiao, X., Zhou, Y., & Gong, Y. J. (2018). RGB-D saliency detection with pseudo depth. IEEE Transactions on Image Processing, 28(5), 2126–2139.
Xie, Y., Lu, H., & Yang, M. H. (2012). Bayesian saliency via low and mid level cues. IEEE Transactions on Image Processing, 22(5), 1689–1698.
Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1155–1162
Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. IEEE Conference on Computer Vision and Pattern Recognition pp 3166–3173
Yang KF, Gao X, Zhao JR, Li YJ (2015) Segmentation-based salient object detection. In: CCF Chinese Conference on Computer Vision, pp 94–102
Yang, K. F., Li, H., Li, C. Y., & Li, Y. J. (2016). A unified framework for salient structure detection by contour-guided visual search. IEEE Transactions on Image Processing, 25(8), 3475–3488.
Yin L, Hou X, Koch C, Rehg JM, Yuille AL (2014) The secrets of salient object segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 280–287
Yuan, Y., Li, C., Kim, J., Cai, W., & DD F. (2017). Reversion correction and regularized random walk ranking for saliency detection. IEEE Transaction Image Process, 27(3), 1–1.
Zeng Y, Zhuge Y, Lu H, Zhang L, Qian M, Yu Y (2019) Multi-source weak supervision for saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 6074–6083
Zhang J, Sclaroff S (2013) Saliency detection: A boolean map approach. In: Proceedings of the IEEE international conference on computer vision, pp 153–160
Zhang, J., & Sclaroff, S. (2015). Exploiting surroundedness for saliency detection: a boolean map approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(5), 889–902.
Zhang, L., Tong, M. H., Marks, T. K., Shan, H., & Cottrell, G. W. (2008). Sun: A bayesian framework for saliency using natural statistics. Journal of vision, 8(7), 32–32.
Zhang L, Zhang J, Lin Z, Lu H, He Y (2019) Capsal: Leveraging captioning to boost semantics for salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 6024–6033
Zhang X, Wang T, Qi J, Lu H, Wang G (2018) Progressive attention guided recurrent network for salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 714–722
Zhao, Q., & Koch, C. (2013). Learning saliency-based visual attention : A review. Signal Processing, 93(6), 1401–1407.
Zhao R, Ouyang W, Li H, Wang X (2015) Saliency detection by multi-context deep learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1265–1274
Zhao T, Wu X (2019) Pyramid feature attention network for saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3085–3094
Zhou, L., & Gu, X. (2020). Embedding topological features into convolutional neural network salient object detection. Neural Networks, 121, 308–318.
Zhu W, Liang S, Wei Y, Sun J (2014) Saliency optimization from robust background detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2814–2821
Zhuo, Y., Zhou, T. G., Rao, H. Y., Wang, J. J., Meng, M., Chen, M., et al. (2003). Contributions of the visual ventral pathway to long-range apparent motion. Science, 299(5605), 417–420.
Zitnick CL, Dollár P (2014) Edge boxes: Locating object proposals from edges. In: European conference on computer vision, Springer, pp 391–405
Acknowledgements
The authors would like to thank Professor Lin Chen for his helpful discussions and suggestions on the modeling of his topological perception theory. This work was supported by the Key Area R&D Program of Guangdong Province (#2018B030338001), the Natural Science Foundations of China (#62076055, #61806041). This work was also supported by the 111 Project (B12027) of China. We also thank LetPub for its linguistic assistance during the preparation of this manuscript.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Jiaya Jia.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Peng, P., Yang, KF., Luo, FY. et al. Saliency Detection Inspired by Topological Perception Theory. Int J Comput Vis 129, 2352–2374 (2021). https://doi.org/10.1007/s11263-021-01478-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-021-01478-4