Skip to main content

Advertisement

SpringerLink
Log in

Segmentation of dermoscopy images based on deformable 3D convolution and ResU-NeXt + +

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Melanoma is one of the most dangerous skin cancers. The current melanoma segmentation is mainly based on FCNs (fully connected networks) and U-Net. Nevertheless, these two kinds of neural networks are prone to parameter redundancy, and the gradient of neural networks disappears that occurs when the neural network backpropagates as the neural network gets deeper, which will reduce the Jaccard index of the skin lesion image segmentation model. To solve the above problems and improve the survival rate of melanoma patients, an improved skin lesion segmentation model based on deformable 3D convolution and ResU-NeXt++ (D3DC- ResU-NeXt++) is proposed in this paper. The new modules in D3DC-ResU-NeXt++ can replace ordinary modules in the existing 2D convolutional neural networks (CNNs) that can be trained efficiently through standard backpropagation with high segmentation accuracy. In particular, we introduce a new data preprocessing method with dilation, crop operation, resizing, and hair removal (DCRH), which improves the Jaccard index of skin lesion image segmentation. Because rectified Adam (RAdam) does not easily fall into a local optimal solution and can converge quickly in segmentation model training, we also introduce RAdam as the training optimizer. The experiments show that our model has excellent performance on the segmentation of the ISIC2018 Task I dataset, and the Jaccard index achieves 86.84%. The proposed method improves the Jaccard index of segmentation of skin lesion images and can also assist dermatological doctors in determining and diagnosing the types of skin lesions and the boundary between lesions and normal skin, so as to improve the survival rate of skin cancer patients.

Graphical abstract

Overview of the proposed model. An improved skin lesion segmentation model based on deformable 3D convolution and ResU-NeXt++ (D3DC- ResU-NeXt++) is proposed in this paper. D3DC-ResU-NeXt++ has strong spatial geometry processing capabilities, it is used to segment the skin lesion sample image; DCRH and transfer learning are used to preprocess the data set and D3DC-ResU-NeXt++ respectively, which can highlight the difference between the lesion area and the normal skin, and enhance the segmentation efficiency and robustness of the neural network; RAdam is used to speed up the convergence speed of neural network and improve the efficiency of segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Schadendorf D, van Akkooi ACJ, Berking C, Griewank KG, Gutzmer R, Hauschild A, Stang A, Roesch A, Ugurel S (2018) Melanoma. Lancet 392:971–984. https://doi.org/10.1016/s0140-6736(18)31559-9

    Article  PubMed  Google Scholar 

  2. Lucas RM, Yazar S, Young AR, Norval M, de Gruijl FR, Takizawa Y, Rhodes LE, Sinclair CA, Neale RE (2019) Human health in relation to exposure to solar ultraviolet radiation under changing stratospheric ozone and climate. Photochem Photobiol Sci 18:641–680. https://doi.org/10.1039/c8pp90060d

    Article  PubMed  CAS  Google Scholar 

  3. Hu L, Jin SF, Chen L, Wang YL (2018) Trends in the incidence and mortality of cutaneous melanoma in Hong Kong between 1983 and 2015. Int J Clin Exp Med 11:8259–8266

    Google Scholar 

  4. Fong ZV, Tanabe KK (2014) Comparison of melanoma guidelines in the USA, Canada, Europe, Australia and New Zealand: a critical appraisal and comprehensive review. Br J Dermatol 170:20–30. https://doi.org/10.1111/bjd.12687

    Article  PubMed  CAS  Google Scholar 

  5. Kasmi R, Mokrani K (2016) Classification of malignant melanoma and benign skin lesions: implementation of automatic ABCD rule. IET Image Process 10:448–455. https://doi.org/10.1049/iet-ipr.2015.0385

    Article  Google Scholar 

  6. Celebi ME, Kingravi HA, Uddin B, Iyatomi H, Aslandogan YA, Stoecker WV, Moss RH (2007) A methodological approach to the classification of dermoscopy images. Comput Med Imaging Graph 31:362–373. https://doi.org/10.1016/j.compmedimag.2007.01.003

    Article  PubMed  PubMed Central  Google Scholar 

  7. Wu J, Chen EZ, Rong R, Li X, Xu D, Jiang H (2019) Skin Lesion Segmentation with C-UNet. In: 2019 41st Annual International Conference of the IEEE. Eng Med Bio Soc:2785–2788. https://doi.org/10.1109/EMBC.2019.8857773

  8. Yang D, Salciccioli J, Marshall D, Sheri A, Shalhoub J (2020) Trends in malignant melanoma mortality in 31 countries from 1985 to 2015. Br J Dermatol 183:1056–1064. https://doi.org/10.1111/bjd.19010

    Article  PubMed  CAS  Google Scholar 

  9. Ma L, Shuai R, Ran X, Liu W, Ye C (2020) Combining DC-GAN with ResNet for blood cell image classification. Med Biol Eng Comput 58:1–14. https://doi.org/10.1007/s11517-020-02163-3

    Article  Google Scholar 

  10. Young K, Booth G, Simpson B, Dutton R, Shrapnel S (2019) Deep neural network or dermatologist? In: Interpretability of machine intelligence in medical image computing and multimodal learning for clinical decision support. Springer, pp 48-55. https://doi.org/10.1007/978-3-030-33850-3_6

  11. Chan S, Reddy V, Myers B, Thibodeaux Q, Brownstone N, Liao W (2020) Machine learning in dermatology: current applications, opportunities, and limitations. Dermatol Ther 10:1–22. https://doi.org/10.1007/s13555-020-00372-0

    Article  Google Scholar 

  12. Mohammed ZF, Abdulla AA (2021) An efficient CAD system for ALL cell identification from microscopic blood images. Multimed Tools Appl 80:6355–6368

    Article  Google Scholar 

  13. Mohammed ZF, Abdulla AA (2020) Thresholding-based white blood cells segmentation from microscopic blood images. J Sci Technol 4:9–17

    Google Scholar 

  14. Jain S, Pise N (2015) Computer aided melanoma skin cancer detection using image processing. Proc Comput Sci 48:735–740. https://doi.org/10.1016/j.procs.2015.04.209

    Article  Google Scholar 

  15. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention:234–241. https://doi.org/10.1007/978-3-319-24574-4_28

  16. Zhang N, Cai Y-X, Wang Y-Y, Tian Y-T, Wang X-L, Badami B (2020) Skin cancer diagnosis based on optimized convolutional neural network. Artif Intell Med 102:101756. https://doi.org/10.1016/j.artmed.2019.101756

    Article  PubMed  Google Scholar 

  17. Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y (2017) Deformable convolutional networks. In: Proceedings of the IEEE international conference on computer vision. 2017:7645–773

  18. Oh CM, Cho H, Won YJ, Kong HJ, Roh YH, Jeong KH, Jung KW (2018) Nationwide trends in the incidence of melanoma and non-melanoma skin cancers from 1999 to 2014 in South Korea. Cancer Res Treat 50:729–737. https://doi.org/10.4143/crt.2017.166

    Article  PubMed  Google Scholar 

  19. Gardner LJ, Strunck JL, Wu YP, Grossman D (2019) Current controversies in early-stage melanoma questions on incidence, screening, and histologic regression. J Am Acad Dermatol 80:1–12. https://doi.org/10.1016/j.jaad.2018.03.053

    Article  PubMed  Google Scholar 

  20. Siegel RL, Miller KD, Jemal A (2019) Cancer statistics, 2019. CA Cancer J Clin 69:7–34. https://doi.org/10.3322/caac.21551

    Article  PubMed  Google Scholar 

  21. Feigelson HS, Powers JD, Kumar M, Carroll NM, Pathy A, Ritzwoller DP (2019) Melanoma incidence, recurrence, and mortality in an integrated healthcare system: a retrospective cohort study. Cancer Med 8:4508–4516. https://doi.org/10.1002/cam4.2252

    Article  PubMed  PubMed Central  Google Scholar 

  22. Duarte AF, Sousa-Pinto B, Freitas A, Delgado L, Costa-Pereira A, Correia O (2018) Skin cancer healthcare impact: a nation-wide assessment of an administrative database. Cancer Epidemiol 56:154–160. https://doi.org/10.1016/j.canep.2018.08.004

    Article  PubMed  CAS  Google Scholar 

  23. Duan Y, Wang J, Hu M, Zhou M, Li Q, Sun L, Qiu S, Wang Y (2019) Leukocyte classification based on spatial and spectral features of microscopic hyperspectral images. Opt Laser Technol 112:530–538. https://doi.org/10.1016/j.optlastec.2018.11.057

    Article  CAS  Google Scholar 

  24. Liu L, Jiang H, He P, Chen W, Liu X, Gao J, Han J (2019) On the variance of the adaptive learning rate and beyond. arXiv. arXiv:190803265

  25. Keskar NS, Socher R (2017) Improving generalization performance by switching from adam to sgd. arXiv. arXiv:171207628

  26. Hua K-L, Hsu C-H, Hidayati SC, Cheng W-H, Chen Y-J (2015) Computer-aided classification of lung nodules on computed tomography images via deep learning technique. OncoTargets Ther:8. https://doi.org/10.2147/OTT.S80733

  27. Mahbod A, Tschandl P, Langs G, Ecker R, Ellinger I (2020) The effects of skin lesion segmentation on the performance of dermatoscopic image classification. Comput Methods Prog Biomed 197:105725

    Article  Google Scholar 

  28. Ballerini L, Fisher RB, Aldridge B, Rees J (2013) A color and texture based hierarchical K-NN approach to the classification of non-melanoma skin lesions. In: Color medical image analysis. Springer, pp 63-86. https://doi.org/10.1007/978-94-007-5389-1_4

  29. Mishra NK, Celebi ME (2016) An overview of melanoma detection in dermoscopy images using image processing and machine learning. arXiv. arXiv:160107843

  30. Cheng Y, Swamisai R, Umbaugh SE, Moss RH, Stoecker WV, Teegala S, Srinivasan SK (2008) Skin lesion classification using relative color features. Skin Res Technol 14:53–64. https://doi.org/10.1111/j.1600-0846.2007.00261.x

    Article  PubMed  Google Scholar 

  31. Jha D, Smedsrud PH, Riegler MA, Johansen D, De Lange T, Halvorsen P, Johansen HD (2019) ResUNet++: an advanced architecture for medical image segmentation. In: 2019 IEEE International Symposium on Multimedia (ISM), pp 225–2255. https://doi.org/10.1109/ISM46123.2019.00049

  32. Huang X (2017) Belongie S Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1501–1510

  33. Jing Y, Liu X, Ding Y, Wang X, Wen S (2020) Dynamic instance normalization for arbitrary style transfer. Proceedings of the AAAI Conference on Artificial Intelligence 34:4369–4376

    Article  Google Scholar 

  34. Alom MZ, Hasan M, Yakopcic C, Taha TM, Asari VK (2018) Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv:180206955

  35. Zhang L, Yang G, Ye X (2019) Automatic skin lesion segmentation by coupling deep fully convolutional networks and shallow network with textons. J Med Imaging 6:024001. https://doi.org/10.1117/1.JMI.6.2.024001

    Article  Google Scholar 

  36. Ali A-R, Li J, O’Shea SJ, Yang G, Trappenberg T, Ye X A (2019) Deep learning based approach to skin lesion border extraction with a novel edge detector in dermoscopy images. In: 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, pp 1-7 https://doi.org/10.1109/IJCNN.2019.8852134

  37. Yu L, Chen H, Dou Q, Qin J, Heng P-A (2016) Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36:994–1004. https://doi.org/10.1109/TMI.2016.2642839

    Article  PubMed  Google Scholar 

  38. Zhang Z, Liu Q, Wang Y (2017) Road extraction by deep residual U-Net. IEEE Geoence and. Remote Sens Lett:1–5

  39. He K, Zhang X, Ren S (2016) Sun J Deep residual learning for image recognition. In: IEEE Conference on Computer Vision & Pattern Recognition

  40. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2018) Unet++: A nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, pp 3-11. https://doi.org/10.1007/978-3-030-00889-5_1

  41. Xie S, Girshick R, Dollár P, Tu Z, He K (2016) Aggregated residual transformations for deep neural networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016

  42. Jie H, Li S, Samuel A, Gang S, Enhua W (2019) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell

  43. Jia X, De Brabandere B, Tuytelaars T, Van Gool L (2016) Dynamic filter networks[J]. Advances in neural information processing systems 29:667–675

  44. Tschandl P, Rosendahl C, Kittler H (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data 5:180161. https://doi.org/10.1038/sdata.2018.161

    Article  PubMed  PubMed Central  Google Scholar 

  45. Codella N, Rotemberg V, Tschandl P, Celebi ME, Dusza S, Gutman D, Helba B, Kalloo A, Liopyris K, Marchetti M (2019) Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration. arXiv:190203368

  46. Telea A (2004) An image inpainting technique based on the fast marching method. J Graph Tools 9:23–34. https://doi.org/10.1080/10867651.2004.10487596

    Article  Google Scholar 

  47. Wang G, Wang Y, Li H, Chen X, Lu H, Ma Y, Peng C, Wang Y, Tang L (2014) Morphological background detection and illumination normalization of text image with poor lighting. PLoS ONE 9:e110991. https://doi.org/10.1371/journal.pone.0110991

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  48. Thapar S, Garg S (2012) Study and implementation of various morphology based image contrast enhancement techniques. CSE Department, IT Department, pp 2–5

    Google Scholar 

  49. Arora R, Raman B, Nayyar K, Awasthi R Automated skin lesion segmentation using attention-based deep convolutional neural network. Biomed Signal Process Control 65:102358

  50. Zhu L, Feng S, Zhu W, Chen X (2020) ASNet: An adaptive scale network for skin lesion segmentation in dermoscopy images. In: Medical Imaging 2020: Biomedical Applications in Molecular, Structural, and Functional Imaging. International Society for Optics and Photonics 113170W https://doi.org/10.1117/12.2549178

  51. Jafari M, Auer D, Francis S, Garibaldi J, Chen X (2020) DRU-Net: an efficient deep convolutional neural network for medical image segmentation. In: 2020 IEEE 17th International Symposium on Biomedical Imaging, pp 1144-1148. https://doi.org/10.1109/ISBI45749.2020.9098391

  52. Saini S, Gupta D, Tiwari AK (2020) Detector-SegMentor network for skin lesion localization and segmentation. arXiv 200506550

  53. Ibtehaz N, Rahman MS (2020) MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw 121:74–87. https://doi.org/10.1016/j.neunet.2019.08.025

    Article  PubMed  Google Scholar 

  54. Al Nazi Z, Abir TA (2020) Automatic skin lesion segmentation and melanoma detection: transfer learning approach with U-Net and DCNN-SVM. In: Proceedings of International Joint Conference on Computational Intelligence. Springer, pp 371-381. https://doi.org/10.1007/978-981-13-7564-4_32

  55. Jha D, Riegler MA, Johansen D, Halvorsen P, Johansen HD (2020) DoubleU-Net: a deep convolutional neural network for medical image segmentation. arXiv 200604868

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China no. 61701222.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Nanjing Tech University, Nanjing, 211816, China

    Chen Zhao, Renjun Shuai & Menglin Wu

  2. Nanjing Health Information Center, Nanjing, 210003, China

    Li Ma

  3. Changzhou No. 2 People’s Hospital affiliated with Nanjing Medical University, Changzhou, 213003, China

    Wenjia Liu

Authors
  1. Chen Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Renjun Shuai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenjia Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Menglin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

C.Z. contributed to the writing and editing of the paper and the operation and editing of the code. R.S. (corresponding author) contributed to technological guidance, provided experimental equipment and major financial support. L.M. contributed technical support and guidance for the paper concept. W.L. contributed to the technical guidance and as a consultant in the medical consultant field, D.H. contributed part of the paper correction, and M.W. contributed to the direction of the paper and the funding of support and related work.

Corresponding author

Correspondence to Renjun Shuai.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, C., Shuai, R., Ma, L. et al. Segmentation of dermoscopy images based on deformable 3D convolution and ResU-NeXt + +. Med Biol Eng Comput 59, 1815–1832 (2021). https://doi.org/10.1007/s11517-021-02397-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-021-02397-9

Keywords

Search

Navigation