Skip to main content
SpringerLink
Log in

Automatic image co-segmentation: a survey

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Image co-segmentation is important for its advantage of alleviating the ill-pose nature of image segmentation through exploring the correlation between related images. Many automatic image co-segmentation algorithms have been developed in the last decade, which are investigated comprehensively in this paper. We firstly analyze visual/semantic cues for guiding image co-segmentation, including object cues and correlation cues. Then, we describe the traditional methods in three categories of object elements based, object regions/contours based, common object model based. In the next part, deep learning-based methods are reviewed. Furthermore, widely used test datasets and evaluation criteria are introduced and the reported performances of the surveyed algorithms are compared with each other. Finally, we discuss the current challenges and possible future directions and conclude the paper. Hopefully, this comprehensive investigation will be helpful for the development of image co-segmentation technique.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://chenlab.ece.cornell.edu/projects/touch-coseg/.

  2. http://www.research.microsoft.com/en-us/projects/objectclassrecognition/.

  3. http://www.msri.org/people/members/eranb/.

  4. http://people.csail.mit.edu/mrub/ObjectDiscovery/.

  5. http://www.image-net.org/.

  6. http://host.robots.ox.ac.uk/pascal/VOC/.

  7. http://www.robots.ox.ac.uk/~vgg/data/flowers/.

References

  1. Rother, C., Minka, T., Blake, A., Kolmogorov, V.: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 1, pp. 993–1000. IEEE (2006)

  2. Dai, J., Nian Wu, Y., Zhou, J., Zhu, S.C.: Proceedings of the IEEE International Conference on Computer Vision, pp. 1305–1312 (2013)

  3. Joulin, A., Bach, F., Ponce, J.: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1943–1950. IEEE (2010)

  4. Kim, G., Xing, E.P., Fei-Fei, L., Kanade, T.: 2011 International Conference on Computer Vision, pp. 169–176. IEEE (2011)

  5. Lee, C., Jang, W.D., Sim, J.Y., Kim, C.S.: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3837–3845 (2015)

  6. Yuan, Z.H., Lu, T., Wu, Y.: IJCAI, pp. 3371–3377 (2017)

  7. Han, J., Quan, R., Zhang, D., Nie, F.: A deep spatial contextual long-term recurrent convolutional network for saliency detection. IEEE Trans. Image Process. 27(4), 1639 (2018)

    Article  MathSciNet  Google Scholar 

  8. Liu, Z., Zhu, J., Bu, J., Chen, C.: Multiple convolutional neural networks for multivariate time series prediction. Neurocomputing 135, 107 (2014)

    Article  Google Scholar 

  9. Chang, K.Y., Liu, T.L., Lai, S.H.: CVPR 2011, pp. 2129–2136. IEEE (2011)

  10. Liu, L., Li, K., Liao, X.: Image co-segmentation by co-diffusion. Circuits, Syst., Signal Process. 36(11), 4423 (2017)

    Article  MathSciNet  Google Scholar 

  11. Li, W., Jafari, O.H., Rother, C.: arXiv preprint arXiv:1804.06423 (2018)

  12. Tao, Z., Liu, H., Fu, H., Fu, Y.: Thirty-First AAAI Conference on Artificial Intelligence (2017)

  13. Faktor, A., Irani, M.: Proceedings of the IEEE International Conference on Computer Vision, pp. 1297–1304 (2013)

  14. Taniai, T., Sinha, S.N., Sato, Y.: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4246–4255 (2016)

  15. Zhang, Q., Zhou, J., Wang, Y., Ye, J., Li, B.: BMVC (2014)

  16. Rubinstein, M., Joulin, A., Kopf, J., Liu, C.: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1939–1946 (2013)

  17. Meng, F., Cai, J., Li, H.: Cosegmentation of multiple image groups. Comput. Vis. Image Underst. 146, 67 (2016)

    Article  Google Scholar 

  18. Jerripothula, K.R., Cai, J., Lu, J., Yuan, J.: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3881–3889. IEEE (2017)

  19. Meng, F., Li, H., Liu, G., Ngan, K.N.: Inference with collaborative model for interactive tumor segmentation in medical image sequences. IEEE Trans. Cybern. 43(2), 725 (2013)

    Article  Google Scholar 

  20. Hochbaum, D.S., Singh, V.: 2009 IEEE 12th International Conference on Computer Vision, pp. 269–276. IEEE (2009)

  21. Mukherjee, L., Singh, V., Dyer, C.R.: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2028–2035. IEEE (2009)

  22. Kim, E., Li, H., Huang, X.: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 686–693. IEEE (2012)

  23. Mukherjee, L., Singh, V., Peng, J.: CVPR 2011, pp. 1881–1888. IEEE (2011)

  24. Rubio, J.C., Serrat, J., López, A., Paragios, N.: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 749–756. IEEE (2012)

  25. Liang, X., Zhu, L., Huang, D.S.: Multi-task ranking SVM for image co-segmentation. Neurocomputing 247, 126 (2017)

    Article  Google Scholar 

  26. Jian, S., Ponce, J.: IEEE International Conference on Computer Vision, pp. 3400–3407 (2013)

  27. Chai, Y., Rahtu, E., Lempitsky, V., Van Gool, L., Zisserman, A.: European Conference on Computer Vision, pp. 794–807. Springer (2012)

  28. Zhu, H., Lu, J., Cai, J., Zheng, J., Thalmann, N.M.: IEEE Winter Conference on Applications of Computer Vision, pp. 485–492. IEEE (2014)

  29. Li, Y., Liu, J., Li, Z., Lu, H., Ma, S.: Object co-segmentation via salient and common regions discovery. Neurocomputing 172, 225 (2016)

    Article  Google Scholar 

  30. Vicente, S., Rother, C., Kolmogorov, V.: CVPR 2011, pp. 2217–2224. IEEE (2011)

  31. Wang, F., Huang, Q., Guibas, L.J.: Proceedings of the IEEE International Conference on Computer Vision, pp. 849–856 (2013)

  32. Kim, G., Xing, E.P.: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 837–844. IEEE (2012)

  33. Wang, Z., Liu, R.: Proceedings of the IEEE International Conference on Computer Vision, pp. 393–400 (2013)

  34. Meng, F., Li, H., Liu, G., Ngan, K.N.: Machine vision and application. IEEE Trans. Multimed. 14(5), 1429 (2012)

    Article  Google Scholar 

  35. Borji, A., Itti, L.: Optimal attentional modulation of a neural population. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 185 (2013)

    Article  Google Scholar 

  36. Zhang, D., Fu, H., Han, J., Borji, A., Li, X.: Exploiting the value of the center-dark channel prior for salient object detection. ACM Trans. Intell. Syst. Technol. (TIST) 9(4), 38 (2018)

    Google Scholar 

  37. Fu, H., Xu, D., Lin, S., Liu, J.: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4428–4436 (2015)

  38. Boykov, Y., Veksler, O., Zabih, R.: Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 1, pp. 377–384. IEEE (1999)

  39. Rother, C., Kolmogorov, V., Blake, A.: ACM Transactions on Graphics (TOG), vol. 23, pp. 309–314. ACM (2004)

  40. Shi, J., Malik, J.: Departmental Papers (CIS), p. 107 (2000)

  41. Grady, L.: Random walks for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1768 (2006)

    Article  Google Scholar 

  42. Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147 (2004)

    Article  Google Scholar 

  43. Boykov, Y.Y., Jolly, M.P.: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol. 1, pp. 105–112. IEEE (2001)

  44. Li, L., Liu, Z., Zhang, J.: Unsupervised image co-segmentation via guidance of simple images. Neurocomputing 275, 1650 (2018)

    Article  Google Scholar 

  45. Ma, J., Li, S., Qin, H., Hao, A.: Unsupervised multi-class co-segmentation via joint-cut over \(L_{1}\)-manifold hyper-graph of discriminative image regions. IEEE Trans. Image Process. 26(3), 1216 (2017)

    Article  MathSciNet  Google Scholar 

  46. Cech, J., Matas, J., Perdoch, M.: Efficient sequential correspondence selection by cosegmentation. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1568 (2010)

    Article  Google Scholar 

  47. Wang, C., Zhang, H., Yang, L., Cao, X., Xiong, H.: Multiple semantic matching on augmented N-partite graph for object co-segmentation. IEEE Trans. Image Process. 26(12), 5825 (2017)

    Article  MathSciNet  Google Scholar 

  48. Li, B., Sun, Z., Li, Q., Wu, Y., Hu, A.: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8519–8528 (2019)

  49. Zhang, K., Chen, J., Liu, B., Liu, Q.: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12813–12820 (2020)

  50. Batra, D., Kowdle, A., Parikh, D., Luo, J., Chen, T.: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3169–3176. IEEE (2010)

  51. Shotton, J., Winn, J., Rother, C., Criminisi, A.: European Conference on Computer Vision, pp. 1–15. Springer (2006)

  52. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

  53. Everingham, M., Eslami, S.A., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111(1), 98 (2015)

    Article  Google Scholar 

  54. Nilsback, M.E., Zisserman, A.: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 2, pp. 1447–1454. IEEE (2006)

  55. Milletari, F., Navab, N., Ahmadi, S.A.: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571. IEEE (2016)

  56. Wong, K.C., Moradi, M., Tang, H., Syeda-Mahmood, T.: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 612–619. Springer (2018)

  57. Fidon, L., Li, W., Garcia-Peraza-Herrera, L.C., Ekanayake, J., Kitchen, N., Ourselin, S., Vercauteren, T.: International MICCAI Brainlesion Workshop, pp. 64–76. Springer (2017)

Download references

Author information

Authors and Affiliations

  1. Beijing Lab of Intelligent Information Technology, School of Computer Science, Beijing Institute of Technology, Beijing, People’s Republic of China

    Xiabi Liu & Xin Duan

Authors
  1. Xiabi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xin Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiabi Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Duan, X. Automatic image co-segmentation: a survey. Machine Vision and Applications 32, 74 (2021). https://doi.org/10.1007/s00138-021-01197-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-021-01197-3

Keywords

Search

Navigation