Skip to main content
SpringerLink
Log in

Bag of contour fragments for improvement of object segmentation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Many state-of-the-art shape features have been proposed for the shape recognition task. In this paper, to explore whether a shape feature influences object segmentation, we propose a specific shape feature, Fisher shape (a form of bag of contour fragments), and we combine this with the appearance feature with multiple kernel learning to create a pipeline of object segmentation system. The experimental results on benchmark datasets clearly demonstrate that the pipeline of object segmentation is effective and that the Fisher shape can improve object segmentation with only the appearance feature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Adamek T, O’Connor N (2004) A multiscale representation method for nonrigid shapes with a single closed contour. IEEE Trans Circuits Syst Video Technol 14:742–753

    Article  Google Scholar 

  2. Alajlan N, Rube I E, Kamel M, Freeman G (2009) Shape retrieval using triangle-area representation and dynamic space warping. Pattern Recognit 40:1911–1920

    Article  Google Scholar 

  3. Alexe B, Deselares T, Ferrari V (2010) What is an object?. In: IEEE international conference on computer vision and pattern recognition, pp 73–80

  4. Aliniya P, Razzaghi P (2018) Parametric and nonparametric context models: a unified approach to scene parsing. Pattern Recognit 84:165–181

    Article  Google Scholar 

  5. Arbelaez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916

    Article  Google Scholar 

  6. Arbelaez P, Pont-Tuset J, Barron J, Marques F, Malik J (2014) Multiscale combinatorial grouping. In: IEEE international conference on computer vision and pattern recognition, pp 328–335

  7. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24:509–522

    Article  Google Scholar 

  8. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140

    MATH  Google Scholar 

  9. Carreira J, Sminchisescu C (2010) Constrained parametric min-cuts for automatic object segmentation. In: IEEE international conference on computer vision and pattern recognition, pp 3241–3248

  10. Carreira J, Caseiro R, Batista J, Sminchisescu C (2012) Semantic segmentation with second-order pooling. In: European conference on computer vision, pp 430–443

    Chapter  Google Scholar 

  11. Chatfield K, Lempitsky V, Vedaldi A, Zisserman A (2011) The devil is in the details: an evaluation of recent feature encoding methods. In: British machine vision conference, pp 1–12

  12. Csurka G, Perronnin F (2011) An efficient approach to semantic segmentation. Int J Comput Vis 95:198–212

    Article  MathSciNet  Google Scholar 

  13. Dai J, He K, Sun J (2015) Convolutional feature masking for joint object and stuff segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3992–4000

  14. Dai L, Yang J, Chen L, Li J (2017) Category-specific object segmentation via unsupervised discriminant shape. Pattern Recognit 64(C):202–214

    Article  Google Scholar 

  15. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE international conference on computer vision and pattern recognition, pp 886–893

  16. Dhillon I, Guan Y, Kulis B (2007) Weighted graph cuts without eigenvectors: a multilevel approach. IEEE Trans Pattern Anal Mach Intell 29(11):1944–1957

    Article  Google Scholar 

  17. Endres I, Hoiem D (2010) Category independent object proposals. In: European conference on computer vision, pp 575–588

    Chapter  Google Scholar 

  18. Everingham M, Gool L V, Williams C, Winn J, Zisserman A (2007) The pascal visual objectclasses challenge 2007. http://www.pascalnetwork.org/challenges/VOC

  19. Farabet C, Couprie C, Najman L, LeCun Y (2013) Learning hierarchical features for scene labeling. IEEE Trans Pattern Anal Mach Intell 35(8):1915–1929

    Article  Google Scholar 

  20. Felzenszwalb P, Schwartz J (2007) Hierarchical matching of deformable shapes. In: IEEE conference on computer vision and pattern recognition, pp 1–8

  21. Felzenszwalb P F, McAllester D A, Ramanan D (2008) A discriminatively trained, multiscale, deformable part model. In: IEEE international conference on computer vision and pattern recognition, pp 1–8

  22. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587

  23. Jain A, Vishwanathan S V N, Varma M (2012) Spg-gmkl: generalized multiple kernel learning with a million kernels. In: Proceedings of the ACM SIGKDD conference on knowledge discovery and data mining, pp 750–758

  24. Kamranian Z, Naghsh Nilchi A R, Monadjemi A, Navab N (2018) Iterative algorithm for interactive co-segmentation using semantic information propagation. Appl Intell 48(12):5019–5036

    Article  Google Scholar 

  25. Kim W, Kim Y (2000) Convexity rule for shape decomposition based on discrete contour evolution. Signal Process Image Commun 16:95–102

    Article  Google Scholar 

  26. Krapac J, Šegvić S (2016) Weakly-supervised semantic segmentation by redistributing region scores back to the pixels. In: German conference on pattern recognition, pp 377–388

    Google Scholar 

  27. Ladicky L, Russell C, Kohli P, Torr P (2009) Associative hierarchical CRFs for object class image segmentation. In: IEEE international conference on computer vision, pp 739– 746

  28. Ladicky L, Russell C, Kohli P, Torr P (2009) Graph cut based inference with co-occurence statistics. In: European conference on computer vision, pp 239–253

    Chapter  Google Scholar 

  29. Latecki L, Lakamper R (1999) Convexity rule for shape decomposition based on discrete contour evolution. Comput Vis Image Underst 73:441–454

    Article  Google Scholar 

  30. Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: IEEE international conference on computer vision and pattern recognition, pp 2169–2178

  31. Li F, Carreira J, Sminchisescu C (2010) Object recognition as ranking holistic figure-ground hypotheses. IEEE international conference on computer vision and pattern recognition, pp 1712–1719

  32. Li X, Liu Z, Luo P, Change Loy C, Tang X (2017) Not all pixels are equal: difficulty-aware semantic segmentation via deep layer cascade. In: The IEEE conference on computer vision and pattern recognition, pp 6459–6468

  33. Li Y, Cao G, Yu Q, Li X (2018a) Active contours driven by non-local gaussian distribution fitting energy for image segmentation. Appl Intell 48(12):4855–4870

    Article  Google Scholar 

  34. Li Y, Liu Y, Liu G, Zhai D, Guo M (2018b) Weakly supervised semantic segmentation based on em algorithm with localization clues. Neurocomputing 275:2574–2587

    Article  Google Scholar 

  35. Liu F, Lin G, Shen C (2015) Crf learning with cnn features for image segmentation. Pattern Recognit 48(10):2983– 2992

    Article  Google Scholar 

  36. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: IEEE international conference on computer vision and pattern recognition, pp 3431–3440

  37. Lucchi A, Li Y, Smith K, Fua P (2012) Structured image segmentation using kernelized features. In: European conference on computer vision, pp 400–413

    Chapter  Google Scholar 

  38. Maninis K K, Caelles S, Pont-Tuset J, Van Gool L (2018) Deep extreme cut: from extreme points to object segmentation. In: The IEEE conference on computer vision and pattern recognition, pp 616–625

  39. McNeill G, Vijayakumar S (2006) Hierarchical procrustes matching for shape retrieval. In: IEEE conference on computer vision and pattern recognition, pp 885–894

  40. Mokhtarian F, Abbasi S, Kittler J (1997) Efficient and robust retrieval by shape content through curvature scale space. Series Softw Eng Knowl Eng 8:51–58

    Article  Google Scholar 

  41. Noh H, Hong S, Han B (2015) Learning deconvolution network for semantic segmentation. In: IEEE conference on computer vision and pattern recognition, pp 1520–1528

  42. Papandreou G, Chen L C, Murphy K P, Yuille AL (2015) Weakly- and semi-supervised learning of a deep convolutional network for semantic image segmentation. In: IEEE international conference on computer vision, pp 1742–1750

  43. Parkhi O, Vedaldi A, Jawahar C V, Zisserman A (2011) The truth about cats and dogs. In: IEEE international conference on computer vision, pp 1427–1434

  44. Pathak D, Krahenbuhl P, Darrell T (2015) Constrained convolutional neural networks for weakly supervised segmentation. In: IEEE international conference on computer vision, pp 1796– 1804

  45. Perronnin F, Sánchez J, Mensink T (2010) Improving the fisher kernel for large-scale image classification. In: European conference on computer vision, pp 143–156

    Chapter  Google Scholar 

  46. Pinheiro P O, Collobert R (2015) From image-level to pixel-level labeling with convolutional networks. In: IEEE international conference on computer vision and pattern recognition, pp 1713–1721

  47. Shen F, Gan R, Yan S, Zeng G (2017) Semantic segmentation via structured patch prediction, context crf and guidance crf. In: The IEEE conference on computer vision and pattern recognition, pp 5178–5186

  48. Shi J, Malik J (1997) Normalized cuts and image segmentation. In: IEEE conference on computer vision and pattern recognition, pp 888–905

  49. Shotton J, Winn J, Rother C, Criminisi A (2006) Textonboost: joint appearance, shape and context modeling for multi-class object recognition and segmentation. In: European conference on computer vision, pp 1–15

    Google Scholar 

  50. Shotton J, Johnson M, Cipolla R (2008) Semantic texton forests for image categorization and segmentation. In: IEEE international conference on computer vision and pattern recognition, pp 1–8

  51. Shotton J, Winn J, Rother C, Criminisi A (2009) Textonboost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int J Comput Vis 81:2–23

    Article  Google Scholar 

  52. Siddiqi K, Shokoufandeh A, Dickinson S J, Zucker S W (1999) Shock graphs and shape matching. Int J Comput Vis 35:13–32

    Article  Google Scholar 

  53. van de Sande KEA, Gevers T, Snoek CGM (2010) Evaluating color descriptors for object and scene recognition. IEEE Trans Pattern Anal Mach Intell 32(9):1582–1596

    Article  Google Scholar 

  54. van de Sande KEA, Uijlings J R R, Gevers T, Smeulders A W M (2011) Segmentation as selective search for object recognition. In: IEEE International Conference on Computer Vision, pp 1879–1886

  55. Verbeek J, Triggs B (2007) Region classification with markov field aspect models. IEEE international conference on computer vision and pattern recognition, pp 1–8

  56. Wang L L, Yung N H C (2015) Improved hierarchical conditional random field model for object segmentation. Mach Vis Appl 26(7):1027–1043

    Article  Google Scholar 

  57. Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: IEEE conference on computer vision and pattern recognition, pp 3360–3367

  58. Wang X, Feng B, Bai X, Liu W, Latecki L J (2014) Bag of contour fragments for robust shape classification. Pattern Recognit 47(6):2116–2115

    Article  Google Scholar 

  59. Wu X, Liu X, Chen Y, Shen J, Zhao W (2018) A graph based superpixel generation algorithm. Appl Intell 48(11):4485–4496

    Article  Google Scholar 

  60. Xu C, Liu J, Tang X (2009) 2d shape matching by contour flexibility. IEEE Trans Pattern Anal Mach Intell 31:180–186

    Article  Google Scholar 

  61. Yang L, Meer P, Foran D (2007) Multiple class segmentation using a unified framework over mean-shift patches. In: IEEE international conference on computer vision and pattern recognition, pp 1–8

  62. Zhang D, Lu G (2002) Generic fourier descriptor for shape-based image retrieval. In: IEEE international conference on multimedia and expo, pp 425–428

  63. Zhang K, Zhang W, Zeng S, Xue X (2014) Semantic segmentation using multiple graphs with block-diagonal constraints. In: AAAI conference on artificial intelligence, pp 2867–2873

  64. Zhang H, Dana K, Shi J, Zhang Z, Wang X, Tyagi A, Agrawal A (2018) Context encoding for semantic segmentation. In: The IEEE conference on computer vision and pattern recognition, pp 7151–7160

Download references

Acknowledgments

We thank the anonymous reviewers for their helpful suggestions. This work was supported by the scientific research fund of Jiangsu University of Technology (KYY17022), the Natural Science Fund Project of Colleges in Jiangsu Province (18KJB520013), Zhejiang Provincial the Natural Science Foundation of China (LQ19F020003), National Nature Science Foundation of China (Grant Nos. 61771146, 61806088, 61472166), the Natural Science Fund of Changzhou (CE20175026) and Qing Lan Project of Jiangsu Province.

Author information

Authors and Affiliations

  1. School of Computer Engineering, Jiangsu University of Technology, Changzhou, People’s Republic of China

    Qian Yu, Honghui Fan, Hongjin Zhu & Feiyue Ye

  2. Laboratory of Cognitive Model and Algorithm, Department of Computer Science, Fudan University, Shanghai, People’s Republic of China

    Qian Yu & Hui Wei

  3. School of Information, Zhejiang University of Finance and Economics, Hangzhou, People’s Republic of China

    Chengzhuan Yang

  4. Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University, Shanghai, People’s Republic of China

    Hui Wei

Authors
  1. Qian Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chengzhuan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Honghui Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongjin Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Feiyue Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hui Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qian Yu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, Q., Yang, C., Fan, H. et al. Bag of contour fragments for improvement of object segmentation. Appl Intell 50, 203–221 (2020). https://doi.org/10.1007/s10489-019-01525-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01525-1

Keywords

Search

Navigation