Skip to main content
SpringerLink
Log in

Investigations of butterfly species identification from images in natural environments

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

It has been a challenging problem to identify species of butterflies, especially from images taken in natural environments. Therefore the First international butterfly species recognition competition was organized at the third Data Mining Competition in China in 2018, so as to find good solutions to this challenging problem. The baseline for the competition was based on the Faster R-CNN for it was the latest deep learning algorithm at that time. Nearly all the competition teams chose the Faster R-CNN, or its variations, to solve the problem. But the identification rates were not good enough, and Faster R-CNN is very time consuming. As a result we have been trying to find the most suitable algorithm to solve the butterfly species identification challenge. This paper will present some investigations we have undertaken in this field over the past two years, and show the results we have obtained. We propose a new partition and augmentation technique for the extremely unbalanced ecological butterfly database. We found that RetinaNet is, so far, the best deep learning algorithm to tackle butterfly species identification based on butterfly images taken in natural environments. The best result we obtained was 79.7% in terms of mAP (mean average precision). This is the best result compared to the state-of-the-art studies in this field on the same database so far.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig.1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Chou, I (1998) Classification and identification of Chinese butterflies. Zhengzhou, China.

  2. CCDM2018:The third China data mining competition (the first international butterfly recognition competition (2018). URL http://ccdm2018.sdufe.edu.cn/info/1012/1072.htm

  3. De Vetter S, Vos R (2018) Image analysis for taxonomic identification of Javanese butterflies. bioRxiv pre-print bioRxiv: 408146

  4. Espeland M, Breinholt J et al (2018) A comprehensive and dated phylogenomic analysis of butterflies. Curr Biol 28(5):770-778.e5. https://doi.org/10.1016/j.cub.2018.01.061

    Article  Google Scholar 

  5. Garczyk Ż, Stach S et al (2018) Segmentation of three-dimensional images of the butterfly wing surface. In: Pietka E, Badura P, Kawa J, Wieclawek W (eds) Information technology in biomedicine. Springer, Poland, pp 111–121

    Google Scholar 

  6. Hernandez-Serna A, Jimenez-Segura L (2014) Automatic identification of species with neural networks. PeerJ 2(2):e563. https://doi.org/10.7717/peerj.563

    Article  Google Scholar 

  7. He, K et al (2018) Deep residual learning for image recognition. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 770–778

  8. Horn G, Aodha O et al (2018) The iNaturalist species classification and detection dataset. In Proceedings of the 31st Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 8769–8778

  9. Kang S, Song S, Lee S (2012) Identification of butterfly species with a single neural network system. J Asia-Pacific Entomol 15(3):431–435. https://doi.org/10.1016/j.aspen.2012.03.006

    Article  Google Scholar 

  10. Kang S, Cho J, Lee S (2014) Identification of butterfly based on their shapes when viewed from different angles using an artificial neural network. J Asia-Pacific Entomol 17(2):143–149. https://doi.org/10.1016/j.aspen.2013.12.004

    Article  Google Scholar 

  11. Kaya Y, Kayci L (2014) Application of artificial neural network for automatic detection of butterfly species using color and texture features. Visual Comp 30(1):71–79. https://doi.org/10.1007/s00371-013-0782-8

    Article  Google Scholar 

  12. Kaya Y, Kayci L et al (2014) Evaluation of texture features for automatic detecting butterfly species using extreme learning machine. J Exp Theor Artif Intell 26(2):267–281. https://doi.org/10.1080/0952813X.2013.861875

    Article  Google Scholar 

  13. Kaya Y, Kayci L, Uyar M (2015) Automatic identification of butterfly species based on local binary patterns and artificial neural network. Appl Soft Comp J 28:132–137. https://doi.org/10.1016/j.asoc.2014.11.046

    Article  Google Scholar 

  14. Kartika D, Herumurti D, Yuniarti A (2018) Local binary pattern method and feature shape extraction for detecting butterfly image. Int J Geomate 15(50):127–133

    Google Scholar 

  15. Lin T et al (2014) Microsoft COCO: common objects in context. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer vision–ECCV 2014, 13rd ECCV. Springer, Switzerland, pp 740–755

    Chapter  Google Scholar 

  16. Liu W, Anguelov D et al (2016) SSD: single shot multibox detector. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer Vision–ECCV 2016. 14th ECCV. Amsterdam, The Netherlands, pp 21–37

  17. Lin T, Dollár P et al (2017) Feature Pyramid Networks for Object Detection. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 936–944

  18. Law H, Deng J (2018) Cornernet: Detecting objects as paired keypoints. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer vision–ECCV 2018, 15th ECCV. Springer, Germany, pp 765–781

    Chapter  Google Scholar 

  19. Li F, Xiong Y (2018) Automatic identification of butterfly species based on HoMSC and GLCMoIB. Vis Comp 34(11):1525–1533. https://doi.org/10.1007/s00371-017-1426-1

    Article  Google Scholar 

  20. Lin T, Goyal P et al (2020) Focal loss for dense object detection. IEEE Trans Pattern Anal Mach Intell 42(2):318–327. https://doi.org/10.1109/TPAMI.2018.2858826

    Article  Google Scholar 

  21. Liang B et al (2020) Butterfly detection and classification based on integrated YOLO algorithm. In: Pan J, Lin J, Liang Y, Chu S (eds) Genetic and Evolutionary Computing. 13rd ICGEC. Dalian, China, pp 500–512

  22. Ren S, He K et al (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031

    Article  Google Scholar 

  23. Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 6517–6525

  24. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. arXiv pre-print

  25. Wang J et al (2012) The identification of butterfly families using content-based image retrieval. Biosys Eng 111(1):24–32. https://doi.org/10.1016/j.biosystemseng.2011.10.003

    Article  Google Scholar 

  26. Xie J, Hou Q et al (2018) The automatic identification of butterfly species. J Comp Res Dev 55(8):1609–1618. https://doi.org/10.7544/issn1000-1239.2018.20180181

    Article  Google Scholar 

  27. Xie J et al (2019) A dataset of butterfly ecological images for automatic species identification. China Sci Data 4(3):265

    Google Scholar 

  28. Xie J, Liu R (2019) The study progress of object detection algorithms based on deep learning. J Shaanxi Normal Univ (Natural Science Edition) 47(5):1–9

    MathSciNet  Google Scholar 

  29. Zhou X, Zhuo J, Krähenbühl P (2019) Bottom-up object detection by grouping extreme and center points. In Proceedings of the 32nd IEEE Conference on Computer Vision and Pattern Recognition(CVPR). IEEE, pp 850–859

  30. Zhou X, Wang D, Krähenbühl P (2019) Objects as points. arXiv pre-print.

Download references

Acknowledgements

This work is supported in part by the National Natural Science Foundation of China under Grant No. of 62076159, and 12031010, and is also supported by the Fundamental Research Funds for the Central Universities under Grant No. of GK202105003 and 2018TS078, and the Innovation Funds of Graduate Programs at Shaanxi Normal University under Grant No. of 2015CXS028 and 2016CSY009. We also acknowledge those researchers who published the open source codes for us to use in this research.

Author information

Authors and Affiliations

  1. School of Computer Science, Shaanxi Normal University, Xi’an, 710119, PR China

    Juanying Xie, Yinyuan Lu & Zhaozhong Wu

  2. College of Life Sciences, Shaanxi Normal University, Xi’an, 710119, PR China

    Shengquan Xu

  3. Department of Computer Science, Swansea University, Swansea, SA2 8PP, UK

    Phil W. Grant

Authors
  1. Juanying Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yinyuan Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhaozhong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shengquan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Phil W. Grant
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Juanying Xie or Shengquan Xu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, J., Lu, Y., Wu, Z. et al. Investigations of butterfly species identification from images in natural environments. Int. J. Mach. Learn. & Cyber. 12, 2431–2442 (2021). https://doi.org/10.1007/s13042-021-01322-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01322-8

Keywords

Search

Navigation