Skip to main content
Log in

Combining DC-GAN with ResNet for blood cell image classification

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

In medicine, white blood cells (WBCs) play an important role in the human immune system. The different types of WBC abnormalities are related to different diseases so that the total number and classification of WBCs are critical for clinical diagnosis and therapy. However, the traditional method of white blood cell classification is to segment the cells, extract features, and then classify them. Such method depends on the good segmentation, and the accuracy is not high. Moreover, the insufficient data or unbalanced samples can cause the low classification accuracy of model by using deep learning in medical diagnosis. To solve these problems, this paper proposes a new blood cell image classification framework which is based on a deep convolutional generative adversarial network (DC-GAN) and a residual neural network (ResNet). In particular, we introduce a new loss function which is improved the discriminative power of the deeply learned features. The experiments show that our model has a good performance on the classification of WBC images, and the accuracy reaches 91.7%.

Overview of the proposed method, we use the deep convolution generative adversarial networks (DC-GAN) to generate new samples that are used as supplementary input to a ResNet, the transfer learning method is used to initialize the parameters of the network, the output of the DC-GAN and the parameters are applied the final classification network. In particular, we introduced a modified loss function for classification to increase inter-class variations and decrease intra-class differences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Akiba T, Suzuki S, Fukuda K (2017) Extremely large minibatch sgd: training resnet-50 on imagenet in 15 minutes. arXiv:1711.04325

  2. AL-Dulaimi K, Banks J, Chandran V, Tomeo-Reyes I, Nguyen Thanh K (2018) Classification of white blood cell types from microscope images: techniques and challenges. In: Microscopy science: Last approaches on educational programs and applied research, vol 8. Formatex Research Center

  3. Alreza Z K K, Karimian A (2016) Design a new algorithm to count white blood cells for classification leukemic blood image using machine vision system. In: 2016 6th International Conference on Computer and Knowledge Engineering (ICCKE). IEEE, pp 251–256

  4. Azam B, Qureshi R J, Jan Z, Khattak T A (2014) Color based segmentation of white blood cells in blood photomicrographs using image quantization. Res J Recent Sciences 2277 :2502–9

    Google Scholar 

  5. Bo B, Li Y, Li W, Wang Y, Tong S (2018) Optogenetic excitation of ipsilesional sensorimotor neurons is protective in acute ischemic stroke: a laser speckle imaging study. IEEE Trans. Biomed. Eng. 66(5):1372–1379

    Article  Google Scholar 

  6. Bonilla E V, Chai K M, Williams C (2008) Multi-task gaussian process prediction. In: Advances in neural information processing systems, pp 153–160

  7. Chatap N, Shibu S (2014) Analysis of blood samples for counting leukemia cells using support vector machine and nearest neighbour. IOSR Journal of Computer Engineering (IOSR-JCE) 16(5):79–87

    Article  Google Scholar 

  8. Çiçek Ö, Abdulkadir A, Lienkamp S S, Brox T, Ronneberger O (2016) 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 424–432

  9. Duan Y, Wang J, Hu M, Zhou M, Li Q, Sun L, Qiu S, Wang Y (2019) Leukocyte classification based on spatial and spectral features of microscopic hyperspectral images. Optics & Laser Technology 112:530–538

    Article  CAS  Google Scholar 

  10. Frid-Adar M, Diamant I, Klang E, Amitai M, Goldberger J, Greenspan H (2018) GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 321:321–331

    Article  Google Scholar 

  11. Gadelha M, Maji S, Wang R (2017) 3D shape induction from 2D views of multiple objects. In: 2017 International Conference on 3D Vision (3DV). IEEE, pp 402–411

  12. Ghosh M, Das D, Mandal S, Chakraborty C, Pala M, Maity A K, Pal S K, Ray A K (2010) Statistical pattern analysis of white blood cell nuclei morphometry. In: 2010 IEEE Students Technology Symposium (TechSym). IEEE, pp 59–66

  13. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp 249–256

  14. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680

  15. Gupta D, Arora J, Agrawal U, Khanna A, de Albuquerque V H C (2019) Optimized Binary Bat algorithm for classification of white blood cells. Measurement

  16. He J, Baxter S L, Xu J, Xu J, Zhou X, Zhang K (2019) The practical implementation of artificial intelligence technologies in medicine. Nature Medicine 25(1):30

    Article  CAS  Google Scholar 

  17. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  18. Hua K L, Hsu C H, Hidayati S C, Cheng W H, Chen Y J (2015) Computer-aided classification of lung nodules on computed tomography images via deep learning technique. OncoTargets and therapy, vol 8

  19. Iizuka S, Simo-Serra E, Ishikawa H (2017) Globally and locally consistent image completion. ACM Transactions on Graphics (ToG) 36(4):107

    Article  Google Scholar 

  20. Irvin J, Rajpurkar P, Ko M, Yu Y, Ciurea-Ilcus S, Chute C, Marklund H, Haghgoo B, Ball R, Shpanskaya K, et al. (2019) Chexpert: a large chest radiograph dataset with uncertainty labels and expert comparison. arXiv:1901.07031

  21. Isensee F, Kickingereder P, Wick W, Bendszus M, Maier-Hein K H (2018) No new-net. In: International MICCAI Brainlesion Workshop. Springer, pp 234–244

  22. Jain S, Pise N, et al. (2015) Computer aided melanoma skin cancer detection using image processing. Procedia Computer Science 48:735–740

    Article  Google Scholar 

  23. Jordan M I, Mitchell T M (2015) Machine learning: trends, perspectives, and prospects. Science 349 (6245):255–260

    Article  CAS  Google Scholar 

  24. Joshi M D, Karode A H, Suralkar S (2013) White blood cells segmentation and classification to detect acute leukemia. International Journal of Emerging Trends & Technology in Computer Science (IJETTCS) 2(3):147–151

    Google Scholar 

  25. Kermany D S, Goldbaum M, Cai W, Valentim C C, Liang H, Baxter S L, McKeown A, Yang G, Wu X, Yan F, et al. (2018) Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5):1122–1131

    Article  CAS  Google Scholar 

  26. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  CAS  Google Scholar 

  27. Lee K, Lee H, Lee K, Shin J (2017) Training confidence-calibrated classifiers for detecting out-of-distribution samples. arXiv:1711.09325

  28. Liang G, Hong H, Xie W, Zheng L (2018) Combining convolutional neural network with recursive neural network for blood cell image classification. IEEE Access 6:36, 188–36, 197

    Article  Google Scholar 

  29. Liu Q, Chen S, Soetikno B, Liu W, Tong S, Zhang H F (2017) Monitoring acute stroke in mouse model using laser speckle imaging-guided visible-light optical coherence tomography. IEEE Trans. Biomed. Eng. 65 (10):2136–2142

    Article  Google Scholar 

  30. Mathieu M, Couprie C, LeCun Y (2015) Deep multi-scale video prediction beyond mean square error. arXiv:1511.05440

  31. Pan S, Yang Q (2010) A survey on transfer learning. IEEE Transaction on Knowledge Discovery and Data Engineering 22 (10)

  32. Prinyakupt J, Pluempitiwiriyawej C (2015) Segmentation of white blood cells and comparison of cell morphology by linear and naïve bayes classifiers. Biomedical Engineering Online 14(1):63

    Article  Google Scholar 

  33. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434

  34. Reed S, Akata Z, Yan X, Logeswaran L, Schiele B, Lee H (2016) Generative adversarial text to image synthesis. arXiv:1605.05396

  35. Rezatofighi S H, Soltanian-Zadeh H (2011) Automatic recognition of five types of white blood cells in peripheral blood. Comput. Med. Imaging Graph. 35(4):333–343

    Article  Google Scholar 

  36. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, et al. (2015) Imagenet large scale visual recognition challenge. International Journal of Computer Vision 115(3):211–252

    Article  Google Scholar 

  37. Saraswat M, Arya K (2014) Feature selection and classification of leukocytes using random forest. Medical & Biological Engineering & Computing 52(12):1041–1052

    Article  Google Scholar 

  38. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 815–823

  39. Schwaighofer A, Tresp V, Yu K (2005) Learning Gaussian process kernels via hierarchical Bayes. In: Advances in neural information processing systems, pp 1209–1216

  40. Sharif Razavian A, Azizpour H, Sullivan J, Carlsson S (2014) CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 806–813

  41. Shen J, Lin K, Wang Y, Pan G (2017) Character recognition from trajectory by recurrent spiking neural networks. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, pp 2900–2903

  42. Sinha N, Ramakrishnan A (2003) Automation of differential blood count. In: TENCON 2003. Conference on Convergent Technologies for Asia-Pacific Region, vol 2. IEEE, pp 547–551

  43. Su M C, Cheng C Y, Wang P C (2014) A neural-network-based approach to white blood cell classification. The Scientific World Journal, vol 2014

  44. Supardi N, Mashor M, Harun N, Bakri F, Hassan R (2012) Classification of blasts in acute leukemia blood samples using k-nearest neighbour. In: 2012 IEEE 8th international colloquium on signal processing and its applications. IEEE, pp 461–465

  45. Szegedy C, Ioffe S, Vanhoucke V, Alemi A A (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence

  46. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  47. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

  48. Tai W L, Hu R M, Hsiao H C, Chen R M, Tsai J J (2011) Blood cell image classification based on hierarchical SVM. In: 2011 IEEE International Symposium on Multimedia. IEEE, pp 129–136

  49. Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual attention network for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3156–3164

  50. Wang J L, Li A Y, Huang M, Ibrahim A K, Zhuang H, Ali A M (2018) Classification of white blood cells with PatternNet-fused ensemble of convolutional neural networks (PECNN). In: 2018 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT). IEEE, pp 325–330

  51. Webb E, Lei C, Huang C J, Kobayashi H, Mikami H, Goda K (2018) Exploring the deep feature space of a cell classification neural network. arXiv:1811.06488

  52. Wen Y, Zhang K, Li Z, Qiao Y (2016) A discriminative feature learning approach for deep face recognition. In: European conference on computer vision. Springer, pp 499–515

  53. Xu Q, Peng J, Shen J, Tang H, Pan G (2019) Deep CovDenseSNN: a hierarchical event-driven dynamic framework with spiking neurons in noisy environment. Neural Networks

  54. Xu Q, Qi Y, Yu H, Shen J, Tang H, Pan G (2018) CSNN: an augmented spiking based framework with perceptron-inception. In: IJCAI, pp 1646–1652

  55. Xu Q, Zhang M, Gu Z, Pan G (2019) Overfitting remedy by sparsifying regularization on fully-connected layers of CNNs. Neurocomputing 328:69–74

    Article  Google Scholar 

  56. Yellin F, Haeffele B D, Roth S, Vidal R (2018) Multi-cell detection and classification using a generative convolutional model. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8953–8961

  57. Zhang J, Xie Y, Wu Q, Xia Y (2019) Medical image classification using synergic deep learning. Medical Image Analysis 54:10–19

    Article  Google Scholar 

  58. Zhang J, Xie Y, Xia Y, Shen C (2019) Attention residual learning for skin lesion classification. IEEE Transactions on Medical Imaging

  59. Zhao J, Zhang M, Zhou Z, Chu J, Cao F (2017) Automatic detection and classification of leukocytes using convolutional neural networks. Medical & Biological Engineering & Computing 55(8):1287–1301

    Article  Google Scholar 

Download references

Acknowledgments

I would like to thank the College of Computer Science and Technology of Nanjing Tech University for providing us with a server for running the code. I would like to thank Dr. Wenjia Liu from Nanjing Drum Tower Hospital for her guidance and help with the medical knowledge of this article. Thanks to other authors who contributed to this article.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Renjun Shuai.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, L., Shuai, R., Ran, X. et al. Combining DC-GAN with ResNet for blood cell image classification. Med Biol Eng Comput 58, 1251–1264 (2020). https://doi.org/10.1007/s11517-020-02163-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-020-02163-3

Keywords

Navigation