Skip to main content
SpringerLink
Log in

Research on license plate location and recognition in complex environment

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Aiming at the problems of license plate location difficulty and low character recognition accuracy in complex environments, such as a small number of license plate samples, illumination transformation, changeable weather and motion blur, this paper proposes an end-to-end license plate recognition method to improve the location and recognition accuracy in complex environments. First, the cyclic generative adversarial network is used to synthesize the approximate real license plate image to enrich the training set and solve the problem of data imbalance to facilitate subsequent model training. Second, a MF-RepUnet license plate location method is proposed, which integrates the improved VGG structure and feature pyramid into the U-Net model to improve the feature extraction capability of the network, and effectively solve the problem of missing detection of inclined license plate and small-scale license plate. Finally, the convolutional recurrent neural network is improved to accurately predict the feature sequence through the way of attention mechanism weighting, which solves the problem of blurred semantic structure sequence features caused by image degradation and further improves the accuracy of license plate character recognition. Experiments show that the proposed method can effectively improve the accuracy and efficiency of license plate location and character recognition, and can be applied to license plate recognition in various complex environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Zhu, J.Y., Park, T., Isola, P., et al.: Unpaired image-to-image translation using cycle-consistent adversarial networks. IEEE Int. Conf. Comput. Vision 2017, 2242–2251 (2017)

    Google Scholar 

  2. Lin T.Y., Piotr, D., Girshick R., et al.: Feature pyramid networks for object detection. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 936–944 (2017)

  3. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, 9351: 234–241 (2015)

  4. Ashtari, A.H., Nordin, M.J., Fathy, M.: An Iranian license plate recognition system based on color features. IEEE Trans. Intell. Transp. Syst. 15(4), 1690–1705 (2014)

    Article  Google Scholar 

  5. Ha, P. S., Shakeri, M.: License plate automatic recognition based on edge detection. Proceedings of 2016 artificial intelligence and robotics. Qazvin: IEEE, 170–174 (2016)

  6. Zhengqiang, L.: Application of deep learning algorithm in license plate recognition system. Chengdu: University of Electronic Science and Technology of China. Department of Communication & Information Systems (2016)

  7. Redmon, J., Divvala, S., Girshick, R., et al.: You only look once: unified, real-time object detection. Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 779–788 (2016)

  8. Liu, W., Anguelov, D., Erhan, D., et al.: SSD: single shot multibox detector. Computer Vision— ECCV 2016, 21–37 (2016)

  9. Ren, S., He, K., Girshick, R., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)

    Article  Google Scholar 

  10. Yongxiang, X., Lanfang, D.: License plate recognition in complex background base on HSV space and template matching. J. Graph. 35(04), 585–589 (2014)

    Google Scholar 

  11. Yaping, Z., Jinshan, Q., Chengzhong, Y.: License plate character segmentation method based on improved con-nected domain algorithm. J. Hangzhou Dianzi Univ. (Nat. Sci.) 36(02), 48–51 (2016)

    Google Scholar 

  12. Mai, V. D., Miao, D., Wang, R., Zhang, H.: An improved method for Vietnam license plate location, segmentation and recognition. Proceedings of 2011 International Conference on Computational and Information Sciences, Chengdu: IEEE, 2011: 212–215

  13. Lin, G., Xue, B., Xu, B., Chen, C.: License plate recognition based on mathematical morphology and template matching. Proceedings of 2019 Chinese Automation Congress (CAC), Hangzhou: IEEE, 2019: 405–410

  14. Bagi, S. S., Harshith, C. H., Tuppad, S., Dinu, V., Srikantaswamy, R.: Hybrid algorithms for moving vehicle plate recognition system. Proceedings of 2019 International Conference on Communication and Electronics Systems (ICCES), Coimbatore: IEEE, 2019: 209–214

  15. Yang, Y., Li, D., Duan, Z.: Chinese vehicle license plate recognition using kernel-based extreme learning machine with deep convolutional features. IET Intel. Transport Syst. 12(3), 213–219 (2018)

    Article  Google Scholar 

  16. Shi, B., Bai, X., Yao, C.: An end-to-end trainable neural net-work for image-based sequence recognition and its ap-plication to scene text recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(11), 2298–2304 (2016)

    Article  Google Scholar 

  17. Li, H., Wang, P., Shen, C.: Toward end-to-end car license plate detection and recognition with deep neural networks. IEEE Trans. Intell. Transp. Syst. 20(3), 1126–1136 (2019)

    Article  Google Scholar 

  18. Sérgio, M., Jung C.R.: Real-time Brazilian license plate detection and recognition using deep convolutional neural networks. Proceedings of 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Niteroi: IEEE, 2017: 55–62

  19. Kipf, T.N., Welling M.: Semi-supervised classification with graph convolutional networks. Preprint at arXiv:1609.02907 (2016)

  20. Hong, D., Gao, L., Yao, J., et al.: Graph convolutional networks for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 59(7), 5966–5978 (2020)

    Article  Google Scholar 

  21. Shi W., Rajkumar R.: Point-gnn: Graph neural network for 3d object detection in a point cloud. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1711–1719 (2020)

  22. Chen, Z.M., Wei, X.S., Wang, P., et al.: Multi-label image recognition with graph convolutional networks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, (2019)

  23. Xu, Z., Yang, W., Meng, A., et al.: Towards end-to-end license plate detection and recognition: a large dataset and baseline. Proceedings of European Conference on Computer Vision (ECCV). Cham: Springer, 2018: 255–271

  24. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial networks. Adv. Neural. Inf. Process. Syst. 3, 2672–2680 (2014)

    Google Scholar 

  25. Ding, X., Zhang, X., Ma, N., et al.: Repvgg: Making vgg-style convnets great again. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13733–13742 (2021)

  26. Szegedy, C., Ioffe, S., Vanhoucke, V., et al.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-first AAAI conference on artificial intelligence (2017)

  27. Beijing Zhiyun View Technology Co., Ltd. HyperLPR. https://gitee.com/zeusees/HyperLPR (2018). Accessed 15 Sept 2021

  28. Fid, A., Fw, B., Pc, A., et al.: ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data—ScienceDirect. ISPRS J. Photogramm. Remote. Sens. 162, 94–114 (2020)

    Article  Google Scholar 

  29. lqian. light-LPR. https://github.com/lqian/light-LPR (2019). Accessed 8 Apr 2022

Download references

Acknowledgements

The authors appreciate the support from the Zhejiang Provincial Natural Science Foundation of China (LY20F020015, LY21F020015), Zhejiang Province Key R&D Project(2021C02012) and the National Science Foundation of China (61902345, 61972121, 61902099, 61802101, 61772525), and the Defense Industrial Technology Development Program (No. JCKY2019415C001).

Author information

Authors and Affiliations

  1. Key Laboratory of Complex Systems Modeling and Simulation, School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China

    Hao Yu, Xingqi Wang, Yanli Shao, Feiwei Qin, Bin Chen & Senlin Gong

Authors
  1. Hao Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xingqi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanli Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Feiwei Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Senlin Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanli Shao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, H., Wang, X., Shao, Y. et al. Research on license plate location and recognition in complex environment. J Real-Time Image Proc 19, 823–837 (2022). https://doi.org/10.1007/s11554-022-01225-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-022-01225-z

Keywords

Search

Navigation