Abstract
In the forensic investigation, recognizing and locating the multiple diatom objects in an image is a challenging issue due to the interferences of the highly complex backgrounds. To address this issue, a coarse to fine diatom recognition and a localization framework based on the deep learning network is proposed in this paper. Firstly, the diatom images are obtained by performing the anatomic study on the cadavers. Next, a high definition electron microscope is scanned. Then, a coarse to fine deep learning framework is constructed to locate and recognize the diatom objects. Unlike the existing diatom classification and recognition methods, which used light microscopy with low resolution and completed under a simple backgrounds, our framework utilizes the high definition electron scanning microscopy with much higher resolution and suffers from the complex backgrounds interferences. To demonstrate the effectiveness of the proposed framework, 4 diatom image datasets with different background interference degrees are constructed. Also, 3 computer numerical simulation analysis are performed. They are (1) the limitations of the traditional methods in the diatom recognition, (2) the optimized composition of the training strategies and the network models, and (3) the performance of the proposed framework. The computer numerical simulation results show that the proposed framework achieves a recognition accuracy of 0.852. This is greater than 0.758 achieved by the AlexNet. Moreover, it can overcome the problem of the highly complex backgrounds interferences in the forensic investigation. Furthermore, it can locate and recognize the multiple objects in various diatom images simultaneously.
Similar content being viewed by others
References
Chang L, Duarte MM, Sucar LE, Morales EF (2012) A Bayesian approach for object classification based on clusters of sift local features. Expert Syst Appl 39(2):1679–1686
Chang CC, Lin CJ (2011) A Library for support vector machines. ACM Trans Intell Syst Technol 2(3):1–39
Deng JH, Wang XY, Zhao J, Liu C, Kang XD, Gu GS (2019) Cyclotella recognition of high-resolution electron microscopy with complex backgrounds. Comput Eng Design 40:167–172
Fang WL, Ding LY, Zhong BT, Love PED, Luo HB (2018) Automated detection of workers and heavy equipment on construction sites: A convolutional neural network approach. Adv Eng Inform 37:139–149
Fei-Fei L, Perona P (2005) A Bayesian hierarchical model for learning natural scene categories. 2005 IEEE Comput Soc Conf Comput Vision Patt Recognit (CVPR’05) 2:524–531
Girshick R (2015) Fast r-cnn. In: Proceedings of the ieee international conference on computer vision, pp 1440–1448
Girshick R, Donahue J, Darrell T, et al. (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587
Krause LMK, Koc J, Rosenhahn B, et al. (2020) A fully convolutional neural network for detection and counting of diatoms on coatings after Short-Term field exposure. Environ Technol 54(16):10022–10030
Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inform Process Syst 25:1097–1105
Liu W, Anguelov D, Erhan D, et al. (2016) SSD: Single Shot MultiBox Detector. Computer Vision - ECCV 2016. ECCV 2016. Lecture Notes in Computer Science. Springer, Cham. https://doi.org/10.1007/978-3-319-46448-0_2
Liu WY, Wen YD, Yu ZD, Ment Y (2016) Large-margin softmax loss for convolutional neural networks. ICML 2(3):7
Ludes B, Coste M, North N, Doray S, Tracqui A, Kintz P (1999) Diatom analysis in victim’s tissues as an indicator of the site of drowning. Int J Legal Med 112(3):163–166
Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans Pattern Anal Mach Intell 27(10):1615–1630
Pedraza A, Bueno G, Deniz O, Cristobal G, Blanco S, Borrego-Ramos M (2017) Automated diatom classification (part b): a deep learning approach. Appl Sci 7(5):460
Pedraza A, Bueno G, Deniz O, Ruiz-Santaquiteria J, Sanchez C, Blanco S, Borrego-Ramos M, Olenici A, Cristobal G (2018) Lights and pitfalls of convolutional neural networks for diatom identification. Optics Photonics Digit Technol Imaging Appl 14
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 779–788
Redmon J, Farhadi A (2018) YOLOv3: An Incremental Improvement. arXiv e-prints
Ren SQ, He KM, Girshick R, Sun J (2015) r-cnn: Faster Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149
Simonyan K, Zisserman A (2014) Very Deep convolutional networks for large-scale image recognition. Comput Sci
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958
Uijlings J, Van D, Gevers T, Smeulsers A (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171
Verikas A, Gelzinis A, Bacauskiene M, Olenina I, Olenin S, vaiciukynas E (2012) Phase congruency-based detection of circular objects applied to analysis of phytoplankton images. Patt Recognit 45(4):1659–167
Xie JZ, Luo TW, Dai JW, wang D, Gao Y, Ran S (2013) Research on automatic analysis algorithm of streaming image of red tide algae. Comput Sci 40:293–296
Yu W, Xue Y, Knoops R, et al. (2020) Automated diatom searching in the digital scanning electron microscopy images of drowning cases using the deep neural networks. Int J Legal Med (1). https://doi.org/10.1007/s00414-020-02392-z
Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. European conference on computer vision 818–833
Zhang YD, Wang SH, Sun P, Phillips P (2015) Pathological brain detection based on wavelet entropy and hu moment invariants. Bio-Med Mater Eng 26:S1283–S1290
Zhang Q, Zhang M, Chen T, et al. (2018) Recent advances in convolutional neural network acceleration. Neurocomputing
Zhang S, Zhou Y, Zhang QZ (2016) Application of the word package model in algae classification and recognition. J Beijing Inform Sci Technol Univ 31:28–32
Zhao J, Liu C, Hu S, et al. (2013) Microwave digestion-vacuum filtration-automated scanning electron microscopy as a sensitive method for forensic diatom test. Int J Legal Med 127(2):459–463
Zhong Z, Sun L, Huo Q (2017) Improved localization accuracy by locnet for faster r-cnn based text detection. In: 2017 14th IAPR international conference on document analysis and recognition (ICDAR). IEEE Computer Society, pp 923–928
Zhou Y, Zhang J, Huang J, et al. (2019) Digital whole-slide image analysis for automated diatom test in forensic cases of drowning using a convolutional neural network algorithm. Forens Sci Int 302:109922
Acknowledgements
This work is supported by the Key Areas Research and Development Program of Guangdong Province under Grant 2019B010139002, the project of Guangzhou Science and Technology under Grant 201902020006, 201902020007, 201807010058.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Deng, J., Wei, H., He, D. et al. A coarse to fine framework for recognizing and locating multiple diatoms with highly complex backgrounds in forensic investigation. Multimed Tools Appl 81, 4839–4857 (2022). https://doi.org/10.1007/s11042-021-11169-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-021-11169-4