Skip to main content
SpringerLink
Log in

Content Based Medical Image Retrieval Based on Salient Regions Combined with Deep Learning

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

In traditional text based medical image retrieval system, it is hard to find visually similar images in large medical image database. Content-based image retrieval is developed to retrieve similar images and it is based on visual attributes describing the content of an image. Developing a method for content based medical image retrieval is a challenging task. This paper proposed a new method for content-based medical image retrieval based on salient regions and deep learning. The proposed method includes two stages: an offline task to extract local object features and an online task for content-based image retrieval in database. In first stage, we extract local object features of medical image depending on shape, texture and intensity, and features extracted by deep learning applied in saliency of decomposition. Secondly, we make online task for content-based image retrieval in database. The user gives query image as an input and the system will retrieve n top most similar images by similarity comparison with bag of code words feature values obtained in the first stage. Evaluation of the proposed method is based on Precision and Recall values. Our dataset includes 5 groups of medical images with their quality varying from low to high. With the best medical image quality group, the accuray can be 91.61% for Precision and 89.61% for Recall. Comparing the average values with others methods, the results of the proposed method are more than 2 to 5% better.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Suharjito A, Santika DD (2017) Content based image retrieval using bag of visual words and multiclass support vector machine. ICIC Express Letters 11(10):1479–1488

  2. Ahmad J, Muhammad K, Baik SW (2017) Medical image retrieval with compact binary codes generated in frequency domain using highly reactive convolutional features. J Med Syst 42(2):24

    Article  Google Scholar 

  3. Deselaers T, Keysers D, Ney H (2008) Features for image retrieval: an experimental comparison. Information Retrieval Journal 11(2):77–107

  4. Srivastava P, Khare A (2017) Integration of wavelet transform, local binary patterns and moments for content-based image retrieval. J Vis Commun Image Represent 42:78–103

    Article  Google Scholar 

  5. Avni U, Greenspan H, Konen E, Sharon M, Goldberger J (2011) X-ray categorization and retrieval on the organ and pathology level, using patchbased visual words. IEEE Trans Med Imaging 30:733–746

    Article  Google Scholar 

  6. Haas S, Donner R, Burner A, Holzer M, Langs G (2012) Superpixel-based interest points for effective bags of visual words medical image retrieval. Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics) 7075 LNCS, 58–68

  7. Qayyum A, Muhammad S, Awais M, Majid M (2017) Neurocomputing medical image retrieval using deep convolutional neural network. Neurocomputing 266:8–20

    Article  Google Scholar 

  8. Youssif AA, Darwish AA, Mohamed RA (2010) Content based medical image retrieval based on pyramid structure wavelet. International Journal of Computer Science and Network Security 10(3):157–164

  9. Binh NT, Tuyet VTH, Vinh PC (2014) Ultrasound images denoising based context awareness in bandelet domain. International Conference on Context-Aware Systems and Applications 128:115–124

  10. Le Pennec E, Mallat S (2005) Sparse geometric image representations with bandelets. IEEE Trans Image Process 15:423–438

    Article  MathSciNet  Google Scholar 

  11. Srivastava P, Binh NT, Khare A (2014) Content-based image retrieval using moments of local ternary pattern. Mobile Networks and Applications 19(5):618–625

  12. Srivastava P, Binh NT, Khare A (2013) Content-based image retrieval using moments. In Proc. 2nd International Conference on Context-Aware Systems and Applications, 228–237

  13. Zhang H, Zhang T, Pedrycz W, Zhao C, Miao D (2019) Improved adaptive image retrieval with the use of shadowed sets. Pattern Recogn 90:390–403

    Article  Google Scholar 

  14. Unar S, Wang X, Wang C, Yu W (2019) A decisive content-based image retrieval approach for feature fusion in visual and textual images. Knowl-Based Syst 179:8–20

    Article  Google Scholar 

  15. Papushoy A, Bors AG (2015) Image retrieval based on query by saliency content. Digital Signal Processing Journal, Elsevier 36:156–173

  16. Wu Y, Liu H, Yuan J, Zhang Q (2018) Is visual saliency useful for content-based image retrieval. Multimed Tools Appl 77:13983–14006

    Article  Google Scholar 

  17. Wan J et al (2014) Deep learning for content- based image retrieval. Proceedings of the ACM international conference on multimedia- MM ‘14, 157–166

  18. Ranl Sarltha R, Varghese P, Ganesh Kumar P (2019) Content based image retrieval using deep learning process. Cluster Computing, Springer Nature 22:4187–4200

  19. Bai C, Huang L, Pan X, Zheng J, Chen S (2018) Optimization of deep convolutional neural network for large scale image retrieval. Neurocomputing 303(16):60–67

    Article  Google Scholar 

  20. Wei S, Liao L, Li J, Zheng Q, Yang F, Zhao Y (2019) Saliency inside: learning attentive CNNs for content-based image retrieval. IEEE Trans Image Process 28(9):4580–4593

    Article  MathSciNet  Google Scholar 

  21. Chen Y, Jiang H, Li C, Jia X, Ghamisi P (2016) Deep feature extraction and classification of Hyperspectral images based on convolutional neural networks. IEEE Trans Geosci Remote Sens 54(10):6232–6251

    Article  Google Scholar 

  22. Le P, Mallat S (2005) Bandelet image approximation and compression. SIAM Journal on Multiscale Modeling and Simulation 4(3):992–1039

  23. Cmap SM, Ceremade GP (2008) Orthogonal Bandelet bases for geometric images approximation. Commun Pure Appl Math LXI:1173–1212

    MathSciNet  Google Scholar 

  24. Huang Y, Zhao Y, Zhao Z (2016) Research on the approximation properties of bandelet transform, 2nd International Conference on Materials Engineering and Information Technology Applications, 556–560

  25. Zhao W, Shihong D (2016) Spectral–spatial feature extraction for Hyperspectral image classification: a dimension reduction and deep learning approach. IEEE Trans Geosci Remote Sens 54(8):4544–4554

    Article  Google Scholar 

  26. Lin Y-Z, Nie Z-H, Ma H-W (2017) Structural damage detection with automatic feature-extraction through deep learning. Wiley Online Library, 32(12): 1025–1046

  27. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88

    Article  Google Scholar 

  28. Achanta R, Estrada F, Wils P, Süsstrunk S (2008) In: Gasteratos A, Vincze M, Tsotsos JK (eds) ICVS 2008Salient region detection and segmentation, vol 5008. Springer, Heidelberg, LNCS, pp 66–75

    Google Scholar 

  29. Rosin PL (2009) A simple method for detecting salient regions. Pattern Recogn 42(11):2363–2371

    Article  Google Scholar 

  30. Ao H, Yu N (2011) Edge saliency map detection with texture suppression. In: Proceedings of Sixth International Conference on Image and Graphics, 309–313

  31. He S, Pugeaulty N (2018) Salient region segmentation. In: Computer Vision and Pattern Recognition, 1–6

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology, VNU-HCM, 268 Ly Thuong Kiet Street, District 10, Ho Chi Minh City, Vietnam

    Vo Thi Hong Tuyet & Nguyen Thanh Binh

  2. Vietnam National University Ho Chi Minh City, Linh Trung Ward, Thu Duc District, Ho Chi Minh City, Vietnam

    Vo Thi Hong Tuyet & Nguyen Thanh Binh

  3. Faculty of Information Technology, Ho Chi Minh City Open University, Ho Chi Minh City, Vietnam

    Vo Thi Hong Tuyet

  4. Faculty of Information Technology, Nguyen Tat Thanh University, Ho Chi Minh City, Vietnam

    Nguyen Kim Quoc

  5. Department of Electronics and Communication, University of Allahabad, Allahabad, India

    Ashish Khare

Authors
  1. Vo Thi Hong Tuyet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nguyen Thanh Binh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nguyen Kim Quoc
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ashish Khare
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nguyen Thanh Binh.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tuyet, V.T.H., Binh, N.T., Quoc, N.K. et al. Content Based Medical Image Retrieval Based on Salient Regions Combined with Deep Learning. Mobile Netw Appl 26, 1300–1310 (2021). https://doi.org/10.1007/s11036-021-01762-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-021-01762-0

Keywords

Search

Navigation