Skip to main content

Advertisement

SpringerLink
Log in

Fully Automatic Arteriovenous Segmentation in Retinal Images via Topology-Aware Generative Adversarial Networks

  • Original research article
  • Published:
Interdisciplinary Sciences: Computational Life Sciences Aims and scope Submit manuscript

Abstract

Retinal image contains rich information on the blood vessel and is highly related to vascular diseases. Fully automatic and accurate identification of arteries and veins from the complex background of retinal images is essential for analyzing eye-relevant diseases, and monitoring progressive eye diseases. However, popular methods, including deep learning-based models, performed unsatisfactorily in preserving the connectivity of both the arteries and veins. The results were shown to be disconnected or overlapped by the twos and thus manual calibration was needed to refine the results. To tackle the problem, this paper proposes a topological structure-constrained generative adversarial network (topGAN) to automatically identify and differentiate the arteries and veins from retinal images. The introduced topological structure term can automatically delineate the topological structure properties of retinal blood vessels and greatly improves the vascular connectivity of the entire arteriovenous classification results. We train and evaluate our model on both the AV-DRIVE public available dataset and the CVDG home-owned dataset, which consists of 40 images and 3119 images, respectively. Experiments demonstrate that integrating topological structure constraints can significantly improve the performance of arteriovenous classification. Our method achieves excellent performance with an accuracy of 94.3% on the AV-DRIVE dataset and 93.6% on the CVDG dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Batra A, Singh S, Pang G, Basu S, Jawahar C, Paluri M (2019) Improved road connectivity by joint learning of orientation and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 10385–10393

  2. Beck A, Teboulle M (2009) Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans Image Process 18(11):2419–2434

    Article  PubMed  Google Scholar 

  3. Cai H, Yang Z, Cao X, Xia W, Xu X (2014) A new iterative triclass thresholding technique in image segmentation. IEEE Trans Image Process 23(3):1038–1046

    Article  PubMed  Google Scholar 

  4. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  PubMed  Google Scholar 

  5. Cheung CYL, Zheng Y, Hsu W, Lee ML, Lau QP, Mitchell P, Wang JJ, Klein R, Wong TY (2011) Retinal vascular tortuosity, blood pressure, and cardiovascular risk factors. Ophthalmology 118(5):812–818

    Article  PubMed  Google Scholar 

  6. Dashtbozorg B, Mendonça AM, Campilho A (2014) An automatic graph-based approach for artery/vein classification in retinal images. IEEE Trans Image Process 23(3):1073–1083

    Article  PubMed  Google Scholar 

  7. Frost S, Kanagasingam Y, Sohrabi H, Vignarajan J, Bourgeat P, Salvado O, Villemagne V, Rowe CC, Macaulay SL, Szoeke C et al (2013) Retinal vascular biomarkers for early detection and monitoring of alzheimer’s disease. Transl Psychiatry 3(2):e233

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Ganin Y, Lempitsky V (2014) \(n^ 4\)-fields: neural network nearest neighbor fields for image transforms. In: Asian conference on computer vision, Springer, New York, pp 536–551

  9. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680

  10. Grisan E, Ruggeri A (2003) A divide et impera strategy for automatic classification of retinal vessels into arteries and veins. In: Proceedings of the 25th Annual international conference of the IEEE engineering in medicine and biology society (IEEE Cat. No. 03CH37439), vol 1, pp 890–893. IEEE

  11. Huang F, Dashtbozorg B, ter Haar Romeny BM (2018) Artery/vein classification using reflection features in retina fundus images. Mach Vis Appl 29(1):23–34

    Article  Google Scholar 

  12. Huang Jb, Zhuo E, Li H, Liu L, Cai H, Ou Y (2019) Achieving accurate segmentation of nasopharyngeal carcinoma in mr images through recurrent attention. In: International conference on medical image computing and computer-assisted intervention, Springer, New York, pp 494–502

  13. Klein R, Klein BE, Knudtson MD, Wong TY, Tsai MY (2006) Are inflammatory factors related to retinal vessel caliber?: the beaver dam eye study. Arch Ophthalmol 124(1):87–94

    Article  PubMed  Google Scholar 

  14. Li Q, Zhong S, Chen Z, Zhou X, Yang G, Zhang H, Wang T (2017) A high-speed end-to-end approach for retinal arteriovenous segmentation. In: 2017 10th international congress on image and signal processing, BioMedical engineering and informatics (CISP-BMEI), pp 1–5. IEEE

  15. Li R, Li M, Li J (2019) Connection sensitive attention u-net for accurate retinal vessel segmentation. arXiv preprint arXiv:1903.05558

  16. Luc P, Couprie C, Chintala S, Verbeek J (2016) Semantic segmentation using adversarial networks. arXiv preprint arXiv:1611.08408

  17. Mirsharif Q, Tajeripour F, Pourreza H (2013) Automated characterization of blood vessels as arteries and veins in retinal images. Comput Med Imaging Graph 37(7–8):607–617

    Article  PubMed  Google Scholar 

  18. Mosinska A, Marquez-Neila P, Koziński M, Fua P (2018) Beyond the pixel-wise loss for topology-aware delineation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3136–3145

  19. Niemeijer M, Xu X, Dumitrescu AV, Gupta P, Van Ginneken B, Folk JC, Abramoff MD (2011) Automated measurement of the arteriolar-to-venular width ratio in digital color fundus photographs. IEEE Trans Med Imaging 30(11):1941–1950

    Article  PubMed  Google Scholar 

  20. Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla NY, Kainz B et al. (2018) Attention u-net: learning where to look for the pancreas. arXiv preprint arXiv:1804.03999

  21. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, et al. (2019) Pytorch: an imperative style, high-performance deep learning library. In: Advances in neural information processing systems, pp. 8024–8035

  22. Pei Z, Liu J, Liu M, Zhou W, Yan P, Wen S, Chen Y (2018) Risk-predicting model for incident of essential hypertension based on environmental and genetic factors with support vector machine. Interdiscip Sci Comput Life Sci 10(1):126–130

    Article  Google Scholar 

  23. Pellegrini E, Robertson G, MacGillivray T, van Hemert J, Houston G, Trucco E (2018) A graph cut approach to artery/vein classification in ultra-widefield scanning laser ophthalmoscopy. IEEE Trans Med Imaging 37(2):516–526

    Article  PubMed  Google Scholar 

  24. Qin X, Zhang Z, Huang C, Gao C, Jagersand M (2019) Basnet: boundary-aware salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  25. Qureshi TA, Habib M, Hunter A, Al-Diri B (2013) A manually-labeled, artery/vein classified benchmark for the drive dataset. In: Proceedings of the 26th IEEE international symposium on computer-based medical systems, pp 485–488. IEEE

  26. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, Springer, New York, pp 234–241

  27. Rothaus K, Jiang X, Rhiem P (2009) Separation of the retinal vascular graph in arteries and veins based upon structural knowledge. Image Vis Comput 27(7):864–875

    Article  Google Scholar 

  28. Shi Z, Xie H, Zhang J, Liu J, Gu L (2018) Vessel enhancement based on length-constrained hessian information. In: 2018 24th international conference on pattern recognition (ICPR), pp 2869–2874. IEEE

  29. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  30. Son J, Park SJ, Jung KH (2017) Retinal vessel segmentation in fundoscopic images with generative adversarial networks. arXiv preprint arXiv:1706.09318

  31. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP et al (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  PubMed  Google Scholar 

  32. Wei Y, Wang Z, Xu M (2017) Road structure refined cnn for road extraction in aerial image. IEEE Geosci Remote Sens Lett 14(5):709–713

    Article  Google Scholar 

  33. Welikala R, Foster P, Whincup P, Rudnicka AR, Owen CG, Strachan D, Barman S et al (2017) Automated arteriole and venule classification using deep learning for retinal images from the UK biobank cohort. Comput Biol Med 90:23–32

    Article  CAS  PubMed  Google Scholar 

  34. Xu X, Ding W, Abràmoff MD, Cao R (2017) An improved arteriovenous classification method for the early diagnostics of various diseases in retinal image. Comput Methods Progr Biomed 141:3–9

    Article  Google Scholar 

  35. Zhao Y, Zheng Y, Liu Y, Zhao Y, Luo L, Yang S, Na T, Wang Y, Liu J (2017) Automatic 2-d/3-d vessel enhancement in multiple modality images using a weighted symmetry filter. IEEE Trans Med Imaging 37(2):438–450

    Article  PubMed  Google Scholar 

  36. Zhuo EH, Zhang WJ, Li HJ, Zhang GY, Jing BZ, Zhou J, Cui CY, Chen MY, Sun Y, Liu LZ et al (2019) Radiomics on multi-modalities mr sequences can subtype patients with non-metastatic nasopharyngeal carcinoma (npc) into distinct survival subgroups. Eur Radiol 29(7):3957

    Article  PubMed  Google Scholar 

  37. Zou Q, Xing P, Wei L, Liu B (2019) Gene2vec: gene subsequence embedding for prediction of mammalian n6-methyladenosine sites from mrna. Rna 25(2):205–218

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

This work was partially supported by in part by the Key-Area Research and Development of Guangdong Province under Grant 2020B010166002, the Science and Technology Program of Guangzhou (202002020049), the National Natural Science Foundation of China (61771007), the Health & Medical Collaborative Innovation Project of Guangzhou City(201803010021).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510006, Guangdong, China

    Jingwen Yang, Yu Hu, Guihua Tao & Hongmin Cai

  2. Department of Ophthalmology, Guangdong Eye Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, 510080, Guangdong, China

    Xinran Dong, Qingsheng Peng & Xiaohong Yang

  3. Boston Children’s Hospital, Harvard Medical School, Boston, MA, 02115, USA

    Yangming Ou

Authors
  1. Jingwen Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinran Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qingsheng Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guihua Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yangming Ou
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Hongmin Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xiaohong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongmin Cai.

Ethics declarations

Conflict of interest

All authors have participated in (a) conception and design, or analysis and interpretation of the data; (b) drafting the article or revising it critically for important intellectual content; and (c) approval of the final version. This manuscript has not been submitted to, nor is under review at, another journal or other publishing venue. The authors have no affiliation with any organization with a direct or indirect financial interest in the subject matter discussed in the manuscript. All authors have affiliations with organizations with direct or indirect financial interest in the subject matter discussed in the manuscript.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, J., Dong, X., Hu, Y. et al. Fully Automatic Arteriovenous Segmentation in Retinal Images via Topology-Aware Generative Adversarial Networks. Interdiscip Sci Comput Life Sci 12, 323–334 (2020). https://doi.org/10.1007/s12539-020-00385-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12539-020-00385-5

Keywords

JEL Classification

Search

Navigation