Skip to main content
SpringerLink
Log in

An efficient approach for sub-image separation from large-scale multi-panel images using dynamic programming

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Multi-panel images are increasingly used in research and medical domains for describing complicated situations like results’ comparison in paper; or case depiction of a patient by combining all his medical images into a consolidated image. However, the content based image retrieval (CBIR) systems face the issue of performance decline in terms of poor retrieval accuracy because the individual sub-images of the multi-panel images cannot be accessed during the searching process. Representing multi-panel images in the form of sub-images is a necessary step for improving the retrieval accuracy of CBIR systems. The state-of-the-art multi-panel image segmentation approaches use recursive approach for sub-image separation, which detects the location of the sub-lines of a line in the multi-panel image appearing in its sub-images repeatedly. This characteristic of the available approaches makes the CBIR incapable to provide the intended results to the end users in real time. In this work, a line detection-based method using dynamic programming is proposed for sub-image separation, which detects the position of every line in the multi-panel image only once, instead of several times as in the case of state-of-art approaches. We evaluated the proposed method on a subset of the imageCLEFmed 2013 dataset, containing 1050 images belonging to different groups. The experimental results depict the effectiveness of the proposed method in term of generating the results quickly without losing the accuracy as compare to the state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ali M, Dong L, Akhtar R (2018) Multi-panel medical image segmentation framework for image retrieval system. Multimed Tools Appl 77(16):20271–20295

    Article  Google Scholar 

  2. S Antani, D Demner-Fushman, J Li, BV Srinivasan, GR Thoma (2008). “Exploring use of images in clinical articles for decision support in evidence-based medicine”, proceeding of SPIE, document recognition and retrieval XV, 28

  3. E Apostolova, D You, Z Xue, S Antani, D Demner-Fushman, and GR Thoma (2013). Image retrieval from scientific publications: text and image content processing to separate multipanel figures, J Am Soc Inf Sci Technol, pp. 893–908, 28

  4. Neculai Archip, Rober Rohling, et al. (2005). “Ultrasound image segmentation using spectral clustering”, Volume 31, Issue 11, Pages 1485–1497, November

  5. Aucar JA, Fernandez L, Wagner-Mann C (2007) If a picture is worth a thousand words, what is a trauma computerized tomography panel worth? Am J Surg 194:734–740

    Article  Google Scholar 

  6. Xiaodong Bai, Zhiguo Cao, Zhenghong Yu, Hu Zhu (2011). “Color image segmentation using watershed and Nyström method based spectral clustering ”, seventh international symposium on multispectral image processing and pattern recognition (MIPPR2011), Guilin, China

  7. CN Brown (1996). “100 years of x-rays”, Engineering Science and Education Journal, vol. 5, Issue 3

  8. B Cheng, S Antani, RJ Stanley, D Demner-Fushman, and GR Thoma (2011). Automatic segmentation of subfigure image panels for multimodal biomedical document retrieval, Proceedings of SPIE Electronic Imaging Science and Technology, Document Retrieval and Recognition XVIII

  9. A Chhatkuli, A Foncubierta-Rodriguez, D Markonis, F Meriaudeau, H Muller (2013). “Separating compound figures in journal articles to allow for subfigure classification”, Proceeding of SPIE, Medical Imaging 2013: Advanced PACS-based Imaging Informatics and Therapeutic Applications, vol. 8674

  10. Cooper MS, Sommers-Herivel G, CT CTP, Mc-Carthy MB, Crawford BD, Phillips C (2004) The zebrafish dvd exchange project: a bioinformatics initiative. Methods Cell Biol 77:439–457

    Article  Google Scholar 

  11. Kalpathy Cramer, H Muller et al (2011). The CLEF 2011 medical image retrieval and classification tasks. Working Notes of CLEF 2011 (Cross Language Evaluation Forum), September

  12. Satoshi Tsutsui David J. Crandall (2017). A data driven approach for compound figure separation using convolutional neural networks, computer vision and pattern recognition

  13. García Seco de Herrera Alba and Kalpathy Cramer Jayashree and Demner Fushman Dina and Antani Sameer and Müller, Henning (2013). “Overview of the Image-CLEF 2013 medical tasks”, Working Notes of CLEF 2013 (Cross Language Evaluation Forum)

  14. Alba García Seco de Herrera, Jayashree Kalpathy Cramer, Dina Demner Fushman, Sameer Antani and Henning Müller (2013). Overview of the ImageCLEF 2013 medical tasks, in: CLEF working notes 2013, Valencia

  15. AGS de Herrera, J Kalpathy Cramer et al (2013). Overview of the imageCLEF 2013 medical tasks. Working notes of CLEF 2013, Spain

  16. Demner-Fushman D, Antani S, Simpson M, Thoma GR (2009) Annotation and retrieval of clinically relevant images. Int J Med Inform 78:e59–e67

    Article  Google Scholar 

  17. D Demner-Fushman, S Antani, and GR Thoma (2007). Automatically finding images for clinical decision support [C]. Seventh IEEE International Conference on Data Mining - Workshops

  18. Doulamis AD, Doulamis ND (2004) Optimal content-based video decomposition for interactive video navigation. IEEE Transactions on Circuits and Systems for Video Technology 14(6):757–775

    Article  Google Scholar 

  19. Doulamis ND, Kokkinos P, Varvarigos E (2012) Resource selection for tasks with time requirements using spectral clustering. IEEE Trans Comput 63(2):461–474

    Article  Google Scholar 

  20. SP Gou, X Zhuang, and LC Jiao (2012). Quantum immune fast spectral clustering for SAR image segmentation, IEEE Geoscience and Remote Sensing Letters, volume: 9 , Issue: 1

  21. W Hersh, H Müller, J Kalpathy-Cramer, E Kim, X Zhou (2008). The consolidated imageCLEFmed medical image retrieval task test collection [J]. J Digit Imaging

  22. N Jhanwar, S Chaudhuri, G Seetharaman, B Zavidovique (2004). Content based image retrieval using motif co occurrence matrix [J]. Image Vision Computing

  23. J Kalpathy-Cramer, W Hersh, S Bedrick, H Muller (2008). Query analysis to improve medical image retrieval[C]. Society for Imaging Informatics in Medicine (SIIM) 2008, Seattle, USA, May

  24. J Kalpathy-Cramera, W Hersh (2007). Automatic image modality based classification and annotation to improve medical image retrieval [J]. Studies in Health Technology and Informatics, 1334–1338

  25. Li P, Jiang X et al (2018) Compound image segmentation of published biomedical figures. Bioinformatics 34(7):1192–1199

    Article  Google Scholar 

  26. LD Lopez, J Yu, CO Tudor, CN Arighi, H Huang, K Vijay-Shanker, CH Wu (2012). “Robust segmentation of biomedical figures for image-based document retrieval”, IEEE international conference on Bioinformatics and Biomedicine (BIBM), October, 1–6

  27. Luxburg V (2007) Ulrike. A tutorial on spectral clustering. Statistics and computing April 17:395–416

    Article  Google Scholar 

  28. Ma W-Y, H Zhang (1999). Content-based image indexing and retrieval. Handbook of Multimedia Computing, CRC Press, Palo Alto

  29. H Muller, Kalpathy Cramer et al (2008). Overview of the imageCLEFmed 2008 medical image retrieval task. Evaluating Systems for Multilingual and Multimodal Information Access, Denmark 2009, 512–522

  30. H Muller, Kalpathy Cramer et al (2010). Overview of the clef 2009 medical image retrieval track [C]. Proceedings of the 10th international conference on Cross-language evaluation forum: multimedia experiments. CLEF’09, Berlin, Heidelberg

  31. Muller H et al (2010) ImageCLEF experimental evaluation in visual information retrieval. The Springer International Series on Information Retrieval. Springer, Berlin Heidelberg, pp 277–294

    Book  Google Scholar 

  32. H Muller, et al (2012). Overview of the imageCLEF 2012 medical image retrieval and classification tasks. Working Notes of CLEF 2012 (Cross Language Evaluation Forum), September

  33. Ng Andrew Y, Michael I Jordan, and Yair Weiss (2002). On spectral clustering: Analysis and an algorithm. Advances in neural information processing systems

  34. N Otsu (1979). A threshold selection method from gray-level histograms [J]. IEEE Transactions on Systems, Man, and [Cybernetics

  35. Rui Y, Huang TS (1999) Image retrieval: Current techniques, promising directions and open issues [J]. J Vis Commun Image Represent 10:39–62

    Article  Google Scholar 

  36. Rui Y, Huang S et al (2002) Relevance feedback: a power tool for interactive content-based image retrieval [J]. IEEE Transactions on Circuits and Video Technology, August, pp 644–655

    Google Scholar 

  37. Shin KG, Zheng Q (1991) Scheduling job operations in an automatic assembly line. IEEE T Robotic Autom 7(3):333–341

    Article  Google Scholar 

  38. S Sreedevi, S Sebastian (2012). Content based image retrieval based on database revision[C]. IEEE International Conference on Machine Vision and Image Processing (MVIP), 29–32

  39. Tagare HD, Jaffe C, Duncan J (1997) Medical image databases: A content-based retrieval approach [J]. J Am Med Inform Assoc 4:184–198

    Article  Google Scholar 

  40. Mario Taschwer and Oge Marques (2016). Compound Figure Separation Combining Edge and Band Separator Detection, International conference on multimedia modeling, pp 162–173

  41. Zhang Y, PB Luh, K Yoneda, T Kano and Y Kyoya (1997). Mixed-model assembly line scheduling using the lagrangian relaxation technique. Proceedings of the IEEE International Conference on Control Applications. Hartford, CT, USA, pp: 429–434

  42. Zheng, Xin, et al (2004). Locality preserving clustering for image database. Proceedings of the 12th annual ACM international conference on Multimedia

Download references

Author information

Authors and Affiliations

  1. Department of IT, Hazara University, Dhodial, Pakistan

    Mushtaq Ali & Amanullah Baloch

  2. Institute of Computing and Information Technology, Gomal University, Dera Ismail Khan, Pakistan

    Muhammad Zubair Asghar

Authors
  1. Mushtaq Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Zubair Asghar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amanullah Baloch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mushtaq Ali.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ali, M., Asghar, M.Z. & Baloch, A. An efficient approach for sub-image separation from large-scale multi-panel images using dynamic programming. Multimed Tools Appl 80, 5449–5471 (2021). https://doi.org/10.1007/s11042-020-09950-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09950-y

Keywords

Search

Navigation