Skip to main content
SpringerLink
Log in

Fundus image-based cataract classification using a hybrid convolutional and recurrent neural network

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Cataract is the most prevailing reason for blindness across the globe, which occupies about 4.2% population of the world. Even with the developments in visual sciences, fundus image-based diagnosis is deemed as a gold standard for cataract detection and grading. Though the increase in the workload of ophthalmologists and complexity of fundus images, the results may be subject to intelligence. Therefore, the development of an automatic method for cataract detection is necessary to prevent visual impairment and save medical resources. This paper aims to provide a novel hybrid convolutional and recurrent neural network (CRNN) for fundus image-based cataract classification. The proposed CRNN fuses the advantages of convolution neural network and recurrent neural network to preserve long- and short-term spatial correlation between the patches. Coupled with transfer learning, we adopt AlexNet, GoogLeNet, ResNet and VGGNet to extract multilevel feature representation and to analyse how well these models perform cataract classification. The results demonstrate that the proposed method outperforms state-of-the-art methods with an average accuracy of 0.9739 for four-class cataract classification and provides a compelling reason to be applied for other retinal diseases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Abràmoff, M.D., Garvin, M.K., Sonka, M.: Retinal imaging and image analysis. IEEE Rev. Biomed. Eng. 3, 169–208 (2010)

    Article  Google Scholar 

  2. An, F., Liu, Z.: Facial expression recognition algorithm based on parameter adaptive initialization of CNN and LSTM. Vis. Comput. 36(3), 483–498 (2020)

    Article  Google Scholar 

  3. Awan, R., Koohbanani, N.A., Shaban, M., Lisowska, A., Rajpoot, N.: Context-aware learning using transferable features for classification of breast cancer histology images. In: International Conference Image Analysis and Recognition, Springer, pp. 788–795 (2018)

  4. Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, IEEE, pp. 60–65 (2005)

  5. Cao, L., Li, H., Zhang, Y., Zhang, L., Xu, L.: Hierarchical method for cataract grading based on retinal images using improved haar wavelet. Inf. Fusion 53, 196–208 (2020)

    Article  Google Scholar 

  6. Chorage, S., Khot, S.S.: Detection of diabetic retinopathy and cataract by vessel extraction from fundus images. In: 2017 International conference of Electronics, Communication and Aerospace Technology (ICECA), vol. 1, IEEE, pp. 638–641 (2017)

  7. Dong, Y., Wang, Q., Zhang, Q., Yang, J.: Classification of cataract fundus image based on retinal vascular information. In: International Conference on Smart Health, Springer, pp. 166–173 (2016)

  8. Dong, Y., Zhang, Q., Qiao, Z., Yang, J.J.: Classification of cataract fundus image based on deep learning. In: 2017 IEEE International Conference on Imaging Systems and Techniques (IST), IEEE, pp. 1–5 (2017)

  9. Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115 (2017)

    Article  Google Scholar 

  10. Fan, W., Shen, R., Zhang, Q., Yang, J.J., Li, J.: Principal component analysis based cataract grading and classification. In: 2015 17th International Conference on E-health Networking, Application and Services (HealthCom), IEEE, pp. 459–462 (2015)

  11. Güven, A.: Automatic detection of age-related macular degeneration pathologies in retinal fundus images. Comput. Methods Biomech. Biomed. Eng. 16(4), 425–434 (2013)

    Article  Google Scholar 

  12. Hagos, M.T., Kant, S.: Transfer learning based detection of diabetic retinopathy from small dataset. arXiv:1905.07203 (2019)

  13. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  14. Hoover, A., Kouznetsova, V., Goldbaum, M.: Locating blood vessels in retinal images by piece-wise threshold probing of a matched filter response. In: Proceedings of the AMIA Symposium, American Medical Informatics Association, p. 931 (1998)

  15. Kabbai, L., Abdellaoui, M., Douik, A.: Image classification by combining local and global features. Vis. Comput. 35(5), 679–693 (2019)

    Article  Google Scholar 

  16. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  17. Li, J., Xie, L., Zhang, L., Liu, L., Li, P., Yang, J.j., Wang, Q.: Interpretable learning: a result-oriented explanation for automatic cataract detection. In: International Conference on Frontier Computing, Springer, pp. 296–306 (2018)

  18. Li, J., Xu, X., Guan, Y., Imran, A., Liu, B., Zhang, L., Yang, J.J., Wang, Q., Xie, L.: Automatic cataract diagnosis by image-based interpretability. In: 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), IEEE, pp. 3964–3969 (2018)

  19. Liang, D., Liang, H., Yu, Z., Zhang, Y.: Deep convolutional bilstm fusion network for facial expression recognition. Vis. Comput. 36(3), 499–508 (2020)

    Article  Google Scholar 

  20. Manchalwar, M., Warhade, K.: Detection of cataract and conjunctivitis disease using histogram of oriented gradient. Int. J. Eng. Technol. (IJET) (2017)

  21. Nayak, J., Acharya, R., Bhat, P.S., Shetty, N., Lim, T.C.: Automated diagnosis of glaucoma using digital fundus images. J. Med. Syst. 33(5), 337 (2009)

    Article  Google Scholar 

  22. Nazeri, K., Aminpour, A., Ebrahimi, M.: Two-stage convolutional neural network for breast cancer histology image classification. In: International Conference Image Analysis and Recognition, Springer, pp. 717–726 (2018)

  23. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)

    Article  Google Scholar 

  24. Pascolini, D., Mariotti, S.P.: Global estimates of visual impairment: 2010. Br. J. Ophthalmol. 96(5), 614–618 (2012)

    Article  Google Scholar 

  25. Pizzarello, L., Abiose, A., Ffytche, T., Duerksen, R., Thulasiraj, R., Taylor, H., Faal, H., Rao, G., Kocur, I., Resnikoff, S.: Vision 2020: The right to sight: a global initiative to eliminate avoidable blindness. Arch. Ophthalmol. 122(4), 615–620 (2004)

    Article  Google Scholar 

  26. Qiao, Z., Zhang, Q., Dong, Y., Yang, J.J.: Application of SVM based on genetic algorithm in classification of cataract fundus images. In: 2017 IEEE International Conference on Imaging Systems and Techniques (IST), IEEE, pp. 1–5 (2017)

  27. Ran, J., Niu, K., He, Z., Zhang, H., Song, H.: Cataract detection and grading based on combination of deep convolutional neural network and random forests. In: 2018 International Conference on Network Infrastructure and Digital Content (IC-NIDC), IEEE, pp. 155–159 (2018)

  28. Röhlig, M., Schmidt, C., Prakasam, R.K., Rosenthal, P., Schumann, H., Stachs, O.: Visual analysis of retinal changes with optical coherence tomography. Vis. Comput. 34(9), 1209–1224 (2018)

    Article  Google Scholar 

  29. Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997)

    Article  Google Scholar 

  30. Shen, D., Wu, G., Suk, H.I.: Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221–248 (2017)

    Article  Google Scholar 

  31. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014)

  32. Somasundaram, S., Alli, P.: A machine learning ensemble classifier for early prediction of diabetic retinopathy. J. Med. Syst. 41(12), 201 (2017)

    Article  Google Scholar 

  33. Song, W., Cao, Y., Qiao, Z., Wang, Q., Yang, J.J.: An improved semi-supervised learning method on cataract fundus image classification. In: 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC), vol. 2, IEEE, pp. 362–367 (2019)

  34. Souza, M.B., Medeiros, F.W., Souza, D.B., Garcia, R., Alves, M.R.: Evaluation of machine learning classifiers in keratoconus detection from orbscan ii examinations. Clinics 65(12), 1223–1228 (2010)

    Article  Google Scholar 

  35. Staal, J., Abràmoff, M.D., Niemeijer, M., Viergever, M.A., Van Ginneken, B.: Ridge-based vessel segmentation in color images of the retina. IEEE Trans. Med. Imaging 23(4), 501–509 (2004)

    Article  Google Scholar 

  36. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

  37. Xiong, L., Li, H., Xu, L.: An approach to evaluate blurriness in retinal images with vitreous opacity for cataract diagnosis. J. Healthc. Eng. 2017 (2017)

  38. Xiong, Y., He, Z., Niu, K., Zhang, H., Song, H.: Automatic cataract classification based on multi-feature fusion and SVM. In: 2018 IEEE 4th International Conference on Computer and Communications (ICCC), IEEE, pp. 1557–1561 (2018)

  39. Xu, X., Zhang, L., Li, J., Guan, Y., Zhang, L.: A hybrid global-local representation CNN model for automatic cataract grading. IEEE J. Biomed. Health Inform. (2019)

  40. Yang, J.J., Li, J., Shen, R., Zeng, Y., He, J., Bi, J., Li, Y., Zhang, Q., Peng, L., Wang, Q.: Exploiting ensemble learning for automatic cataract detection and grading. Comput. Methods Programs Biomed. 124, 45–57 (2016)

    Article  Google Scholar 

  41. Zhang, L., Li, J., Han, H., Liu, B., Yang, J., Wang, Q., et al.: Automatic cataract detection and grading using deep convolutional neural network. In: 2017 IEEE 14th International Conference on Networking, Sensing and Control (ICNSC), IEEE, pp. 60–65 (2017)

  42. Zheng, J., Guo, L., Peng, L., Li, J., Yang, J., Liang, Q.: Fundus image based cataract classification. In: 2014 IEEE International Conference on Imaging Systems and Techniques (IST) Proceedings, IEEE, pp. 90–94 (2014)

Download references

Acknowledgements

The authors would like to thank the local hospital (Tongren) for providing data. Further, all procedures performed in studies involving human participants were under the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants involved in the study. The study protocol and consent have been reviewed by the School of Software Engineering, Beijing University of Technology.

Funding

The funding was provided by Beijing Municipal Science and Technology (Grand No. KM201910005028)

Author information

Authors and Affiliations

  1. School of Software Engineering, Beijing University of Technology, Beijing, 100124, China

    Azhar Imran, Jianqiang Li, Faheem Akhtar & Tariq Mahmood

  2. Beijing Engineering Research Center for IoT Software and Systems, Beijing, 100124, China

    Jianqiang Li

  3. Computer Science Division, University of Aizu, Aizuwakamatsu, Fukushima, 965-8580, Japan

    Yan Pei

  4. Department of Computer Science, Sukkur IBA University, Sukkur, 65200, Pakistan

    Faheem Akhtar

  5. Division of Science and Technology, University of Education, Lahore, 54000, Pakistan

    Tariq Mahmood

  6. Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, China

    Li Zhang

Authors
  1. Azhar Imran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianqiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Pei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Faheem Akhtar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tariq Mahmood
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Li Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianqiang Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the Key Program of National Natural Science of China (Grant No. 71432004).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Imran, A., Li, J., Pei, Y. et al. Fundus image-based cataract classification using a hybrid convolutional and recurrent neural network. Vis Comput 37, 2407–2417 (2021). https://doi.org/10.1007/s00371-020-01994-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01994-3

Keywords

Search

Navigation