Skip to main content
SpringerLink
Log in

Optimal Deep Learning based Convolution Neural Network for digital forensics Face Sketch Synthesis in internet of things (IoT)

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

The rapid development in 5G cellular and IoT technologies is expected to be deployed widespread in the next few years. At the same time, crime rates are also increasing to a greater extent while the investigation officers are held responsible to deal with a broad range of cyber and internet issues in investigations. Therefore, advanced IT technologies and IoT devices can be deployed to ease the investigation process, especially, the identification of suspects. At present, only a few research works has been conducted upon deep learning-based Face Sketch Synthesis (FSS) models, concerning its success in diverse application domains including conventional face recognition. This paper proposes a new IoT-enabled Optimal Deep Learning based Convolutional Neural Network (ODL-CNN) for FSS to assist in suspect identification process. The hyper parameter optimization of the DL-CNN model was performed using Improved Elephant Herd Optimization (IEHO) algorithm. In the beginning, the proposed method captures the surveillance videos using IoT-based cameras which are then fed into the proposed ODL-CNN model. The proposed method initially involves preprocessing in which the contrast enhancement process is carried out using Gamma correction method. Then, the ODL-CNN model draws the sketches of the input images following which it undergoes similarity assessment, with professional sketch being drawn as per the directions from eyewitnesses. When the similarity between both the sketches are high, the suspect gets identified. A comprehensive qualitative and quantitative examination was conducted to assess the effectiveness of the presented ODL-CNN model. A detailed simulation analysis pointed out the effective performance of ODL-CNN model with maximum average Peak Signal to Noise Ratio (PSNR) of 20.11dB, Average Structural Similarity (SSIM) of 0.64 and average accuracy of 90.10%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Reyes A, Brittson R, O’Shea K, Steele J (2011) Cybercrime investigations: bridging the gaps between security professionals, law enforcement, and prosecutors. Elsevier, Amsterdam

    Google Scholar 

  2. Brown CS (2015) Investigating and prosecuting cyber crime: forensic dependencies and barriers to justice. Int J Cyber Criminol 9(1):55

    Google Scholar 

  3. Klum S, Han H, Jain AK, Klare B, (2013) Sketch based face recognition: forensic vs. composite sketches. In: 2013 international conference on biometrics (ICB), IEEE, pp 1–8

  4. Wang N, Gao X, Sun L, Li J (2017) Bayesian face sketch synthesis. IEEE Trans Image Process 26(3):1264–1274

    Article  MathSciNet  Google Scholar 

  5. Wang N, Gao X, Sun L, Li J (2018) Anchored neighborhood index for face sketch synthesis. IEEE Trans Circuits Syst Video Technol 28(9):2154-2163

    Google Scholar 

  6. Wang N, Gao X, Li J (2018) Random sampling for fast face sketch synthesis. Pattern Recognit 76:215–227

    Article  Google Scholar 

  7. Wang N, Zhu M, Li J, Song B, Li Z (2017) Data-driven vs. model-driven: fast face sketch synthesis. Neurocomputing 257:214–221

    Article  Google Scholar 

  8. Hwang B-W, Lee S-W (2003) Reconstruction of partially damaged face images based on a morphable face model. IEEE Trans Pattern Anal Mach Intell 25(3):365–372

    Article  Google Scholar 

  9. Mo Z, Lewis J, Neumann U (2004) Face inpainting with local linear representations. In: British Machine Vision Conference (BMVC), pp. 347–356. https://doi.org/10.5244/C.18.37

  10. Wang Z-M, Tao J-H (2007) Reconstruction of partially occluded face by fast recursive PCA. In: Proceedings of the International Conference on Computational Intelligence and Security Workshops (2007) CISW 2007, IEEE, pp 304–307

  11. Burgos-Artizzu XP, Perona P, Dollár P (2013) Robust face landmark estimation under occlusion. In: Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV), IEEE, pp 1513–1520

  12. Mairal J, Bach F, Ponce J, Sapiro G (2009) Online dictionary learning for sparse coding. In: Proceedings of the Twenty-sixth Annual International Conference on Machine Learning, ACM, pp 689–696

  13. Li Y, Liu S, Yang J, Yang M-H (2017) Generative face completion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1, p 6

  14. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680

  15. Mao X, Wang S, Zheng L, Huang Q (2018) Semantic invariant cross-domain image generation with generative adversarial networks. Neurocomputing 293:55–63

    Article  Google Scholar 

  16. Wang T-C, Liu M-Y, Zhu J-Y, Tao A, Kautz J, Catanzaro B, High-resolution image synthesis and semantic manipulation with conditional GANS, arXiv preprint arXiv:1711.11585

  17. Lu D, Chen Z, Wu QJ, Zhang X (2019) FCN based preprocessing for exemplar-based face sketch synthesis. Neurocomputing 365:113–124

    Article  Google Scholar 

  18. Ye L, Zhang B, Yang M, Lian W (2019) Triple-translation GAN with multi-layer sparse representation for face image synthesis. Neurocomputing 358:294–308

    Article  Google Scholar 

  19. Licheng J, Zhang S, Li L, Liu F, Ma W (2018) A modified convolutional neural network for face sketch synthesis. Pattern Recognit 76:125–136

    Article  Google Scholar 

  20. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al. (2016) photo-realistic single image super-resolution using a generative adversarial network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp, 21–26

  21. Lin G, Wu Q, Qiu L, Huang X (2018) Image super-resolution using a dilated convolutional neural network. Neurocomputing 275:1219–1230

    Article  Google Scholar 

  22. Zhou F, Li X, Li Z (2018) High-frequency details enhancing densenet for super-resolution. Neurocomputing 290:34–42

    Article  Google Scholar 

  23. Wang GG, Deb S, Gao XZ, Coelho LDS (2016) A new metaheuristic optimisation algorithm motivated by elephant herding behaviour. Int J Bio-Inspired Comput 8(6):394–409

    Article  Google Scholar 

  24. Li J, Guo L, Li Y, Liu C (2019) Enhancing elephant herding optimization with novel individual updating strategies for large-scale optimization problems. Mathematics 7(5):395

    Article  Google Scholar 

  25. Sketch IIIT-D, Database, http://www.iab-rubric.org/resources/sketchDatabase.html

  26. Wang X, Tang X (2009) Face photo-sketch synthesis and recognition. IEEE Trans Pattern Anal Mach Intell 31(11):1955–1967

    Article  MathSciNet  Google Scholar 

  27. Mart´ınez A, Benavente R (1998) The ar face database. In: Tech. Rep. 24, Computer 565 Vision Center, Bellatera

  28. Zhang W, Wang X, Tang X (2011) Coupled information-theoretic encoding for face photo-sketch recognition. In: Proceedings of the 24th IEEE Conference, Comput. Vis. and Pattern Recognit. (CVPR), pp, 513–520. https://doi.org/10.1109/CVPR.2011.5995324

Download references

Acknowledgements

This article has been written with the financial support of RUSA–Phase 2.0 grant sanctioned vide Letter No. F. 24-51/2014-U, Policy (TNMulti-Gen), Dept. of Edn. Govt. of India, Dt. 09.10.2018.

Author information

Authors and Affiliations

  1. Faculty of Computers and Information, Mansoura University, Dakahlia Governorate, Egypt

    Mohamed Elhoseny

  2. Department of Mathematics, College of Science and Humanities in Alaflaj, Prince Sattam bin Abdulaziz University, Alaflaj, Saudi Arabia

    Mahmoud Mohamed Selim

  3. Department of Computer Applications, Alagappa University, Karaikudi, India

    K. Shankar

Authors
  1. Mohamed Elhoseny
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mahmoud Mohamed Selim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. K. Shankar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. Shankar.

Ethics declarations

Conflict of interest

All the authors of the manuscript declared that there are no potential conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Elhoseny, M., Selim, M.M. & Shankar, K. Optimal Deep Learning based Convolution Neural Network for digital forensics Face Sketch Synthesis in internet of things (IoT). Int. J. Mach. Learn. & Cyber. 12, 3249–3260 (2021). https://doi.org/10.1007/s13042-020-01168-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-020-01168-6

Keywords

Search

Navigation