Skip to main content
SpringerLink
Log in

The Method of Classifying Fog Level of Outdoor Video Images Based on Convolutional Neural Networks

  • Research Article
  • Published:
Journal of the Indian Society of Remote Sensing Aims and scope Submit manuscript

Abstract

The outdoor video has become an important mean of monitoring on-site and target recognition recently, which requires higher and higher quality of the outdoor video images. Foggy weather can cause a significant drop in the quality of outdoor video images, directly affecting the accuracy of monitoring on-site and target recognition. In order to automatically select high-quality images with no fog or mist from the outdoor video, a method of fog level classification in outdoor video images based on convolutional neural networks (CNN) is proposed in this paper. A type of CNN with three convolutional layers, three pooling layers, and three fully connected layers is constructed, and the parameters such as the convolution kernel size, pooling kernel size and padding size are adjusted to obtain the optimal network type. The network can automatically learn the fog concentration characteristics in the image, and can effectively divide the image into three levels: fog-free, mist and dense fog. The method is compared with K-means clustering, support vector machine, and random forest classification. The results show that the constructed CNN has the highest accuracy for fog classification, which can reach more than 99%. Nevertheless, the accuracies of K-means clustering, support vector machine (SVM), and random forest classification are 85.60%, 81.48%, and 89.48%, respectively. With the CNN model constructed in this study for fog level classification, high-quality images with no fog or mist can be automatically selected for outdoor video image analysis and applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Adam, A., & Ioannidis, C. (2014). Automatic road sign detection and classification based on support vector machines and HOG descriptions. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2(5), 1–7.

    Article  Google Scholar 

  • Anwar, M. I., & Khosla, A. (2017). Vision enhancement through single image fog removal. Engineering Science and Technology, an International Journal, 20(3), 1075–1083. https://doi.org/10.1016/j.jestch.2016.11.015

    Article  Google Scholar 

  • Asery, R., Sunkaria, R. K., Sharma, L. D., & Kumar, A. (2016). Fog detection using GLCM based features and SVM. Conference on Advances in Signal Processing (CASP). https://doi.org/10.1109/CASP.2016.7746140

    Article  Google Scholar 

  • Awad, M., & Khanna, R. (2015). Deep learning. Efficient Learning Machines. https://doi.org/10.1007/978-1-4302-5990-9_9

    Article  Google Scholar 

  • Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.

    Article  Google Scholar 

  • Chen, Y. Y., & Feng, H. Z. (2004). A new method for non-liner classify and non-liner regression I: introduction to support vector machine. Journal of Applied Meteorological Science, 15(3), 345–354.

    Google Scholar 

  • Choi, L. K., You, J., & Bovik, A. C. (2014). Reference less perceptual fog density prediction model. Human Vision and Electronic Imaging, 10(1117/12), 2036477.

    Google Scholar 

  • Chu, W. T., Zheng, X. Y., & Ding, S. D. (2017). Camera as weather sensor: Estimating weather information from single images. Journal of Visual Communication and Image Representation, 34(1), 233–249. https://doi.org/10.1016/j.jvcir.2017.04.002

    Article  Google Scholar 

  • Coates, A., Lee, H., & Ng, A.Y. (2011) An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics (pp. 215–223).

  • Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.

    Google Scholar 

  • Du, P. J., Samat, A., Waske, B., Liu, S. C., & Li, Z. H. (2015). Random forest and rotation forest for fully polarized SAR image classification using polarimetric and spatial features. ISPRS Journal of Photogrammetry and Remote Sensing, 105, 38–53. https://doi.org/10.1016/j.isprsjprs.2015.03.002

    Article  Google Scholar 

  • Elhoseiny, M., Huang, S., & Elgammal, A. (2015). Weather classification with deep convolutional neural networks. IEEE International Conference on Image Processing. https://doi.org/10.1109/ICIP.2015.7351424

    Article  Google Scholar 

  • Fadi, A. M., Mouhannad, A., Ahmad, H. M., Christopher, S., Markus, G., & Kyandoghere, K. (2016). Real-time raindrop detection based on cellular neural networks for ADAS. Journal of Real-Time Image Processing, 16, 1–13.

    Google Scholar 

  • Jarrett, K., Kavukcuoglu, K., Ranzato, M.A., & LeCun, Y. (2009) What is the best multi-stage architecture for object recognition. In 2009 IEEE 12th international conference on computer vision (pp. 2146–2153). https://doi.org/10.1109/ICCV.2009.5459469.

  • Ji, G.H. (2018) The Design and implementation of Intelligent Technology Platform. Master Thesis, Beijing Jiaotong University.

  • Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90.

    Article  Google Scholar 

  • Kumar, S. S., Abraham, D. M., Jahanshahi, M. R., Iseley, T., & Starr, J. (2018). Automated defect classification in sewer closed circuit television inspections using deep convolutional neural networks. Automation in Construction, 91, 273–283. https://doi.org/10.1016/j.autcon.2018.03.028

    Article  Google Scholar 

  • Li, G., Wu, J. F., & Lei, Z. Y. (2014a). Research progress of image haze grade evaluation and dehazing technology Laser Journal. Laser Journal, 35(9), 1–6. (in Chinese with English Abstract).

    Google Scholar 

  • Li, K., Chen, H. L., Zhang, S. W., & Wan, J. J. (2018). An SVM based technology for haze image classification. Electronics Opitcs & Control, 25(3), 37-41+.

    Google Scholar 

  • Liu, C.X., Lu, X.B., Ji, S.P., & Geng, W. (2014) A Fog Level Detection Method Based on Image HSV Color Histogram. In 2014 IEEE International Conference on Progress in Informatics and Computing (pp. 373–377). https://doi.org/10.1109/PIC.2014.6972360.

  • Liu, C., Tan, K., Liu, S. H., & Ma, Y. R. (2018). Quality assessment for contrast-distorted images based on convolutional neural network. Microelectronics & Computer., 35(4), 84–88. (in Chinese with English Abstract).

    Google Scholar 

  • Liu, W., Yang, Y., & Wei, L. S. (2017). weather recognition of street scene based on sparse deep neural networks. Journal of Advanced Computational Intelligence and Informatics, 21(3), 403–408.

    Article  Google Scholar 

  • Lu, C. W., Lin, D., Jia, J. Y., & Tang, C. K. (2017). Two-Class Weather Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2510–2524.

    Article  Google Scholar 

  • Luo, Y. M., Huang, D. T., Liu, P. Z., & Feng, H. M. (2016). An novel random forests and its application to the classification of mangroves remote sensing image. Multimedia Tools and Applications, 75(16), 9707–9722.

    Article  Google Scholar 

  • Meng, F., Li, T., Xu, G. L., & Han, Y. D. (2015). Research on method of foggy weather recognition based on K-means clustering algorithm. Modern Electronics Technique, 38(22), 80–83. (in Chinese with English Abstract).

    Google Scholar 

  • Millard, K., & Richardson, M. (2015). On the importance of training data sample selection in random forest image classification: A case study in peatland ecosystem mapping. Remote Sensing, 7(7), 8489–8515. https://doi.org/10.3390/rs70708489

    Article  Google Scholar 

  • Neha, S., Vibhor, J., & Anju, M. (2018). An analysis of convolutional neural networks for image classification. Procedia Computer Science, 132, 377–384. https://doi.org/10.1016/j.procs.2018.05.198

    Article  Google Scholar 

  • Pavlic, M., Belzner, H., Rigoll, G., & Ilic, S. (2012). Image based fog detection in vehicles. IEEE Intelligent Vehicles Symposium. https://doi.org/10.1109/IVS.2012.6232256

    Article  Google Scholar 

  • Sadia, D., Anand, P., Awais, A., Gupta, B., & Seungmin, R. (2018). Service orchestration of optimizing continuous features in industrial surveillance using big data based fog-enabled internet of things. IEEE Access, 6, 21582–21591. https://doi.org/10.1109/ACCESS.2018.2800758

    Article  Google Scholar 

  • Schwarzlmueller, C., Machot, F.A., Fashi, A., & Kyamakya, K. (2011) Adaptive Contrast Enhancement Involving CNN-based Processing for Foggy Weather Conditions & Non-Uniform Lighting Conditions. In Proceedings of the Joint INDS'11 & ISTET'11, 1–10 https://doi.org/10.1109/INDS.2011.6024782.

  • Sermanet, P., & Le Cun, Y. (2011) Traffic sign recognition with multi-scale Convolutional Networks. In The 2011 International Joint Conference on Neural Networks (pp. 2809–2813). https://doi.org/10.1109/IJCNN.2011.6033589.

  • Shi, D., & Yang, X. J. (2016). An assessment of algorithmic parameters affecting image classification accuracy by random forests. Photogrammetric Engineering and Remote Sensing, 82(6), 407–417.

    Article  Google Scholar 

  • Sun, Z. Y., Lu, C. X., Shi, Z. Z., & Gang, M. A. (2016). Research and advances on deep learning. Computer Science, 43(2), 1–8.

    Google Scholar 

  • Sun, Z. C., Zhao, X. W., Wu, M. F., & Wang, C. Z. (2019). Extracting urban impervious surface from worldView-2 and airborne LiDAR data using 3D convolutional neural networks. Journal of the Indian Society of Remote Sensing, 47(3), 401–412. https://doi.org/10.1007/s12524-018-0917-5

    Article  Google Scholar 

  • Wu, M. Y., Chen, L., & Tian, J. (2016). Video image distortion detection and classification based on CNN. Application Research of Computers, 33(9), 2827–2830. (in Chinese with English Abstract).

    Google Scholar 

  • Xie, L. M., Wang, Q. M., & Shi, H. S. (2016). A method of automatic detection of fog image based on SVM classification. Revista De La Facultad De Ingeniería, 31(9), 246–254.

    Google Scholar 

  • Xu, Y., Wen, J., Fei, L. K., & Zhang, Z. (2017). Review of video and image defogging algorithms and related studies on image restoration and enhancement. IEEE Access, 4, 165–188. https://doi.org/10.1109/ACCESS.2015.2511558

    Article  Google Scholar 

  • Yan, G., Yu, M., Shi, S., & Feng, C. (2017). The recognition of traffic speed limit sign in hazy weather. Journal of Intelligent & Fuzzy Systems, 33(2), 873–883. https://doi.org/10.3233/JIFS-162138

    Article  Google Scholar 

  • Yin, H. (2017) Multipurpose image quality assessment for both human and computer vision systems via convolutional neural network (Master's thesis, University of Waterloo).

  • Yu, X. L., Xiao, C. B., Deng, M., & Li, P. (2011). A classification algorithm to distinguish image as haze or non-haze. IEEE Sixth International Conference on Image and Graphics, 8(9), 286–289. https://doi.org/10.1109/ICIG.2011.22

    Article  Google Scholar 

  • Zhang, W. D., Xu, Y. L., Ni, J. C., & Shi, H. H. (2016). Image target recognition method based on multi-scale block convolutional neural network. Journal of Computer Applications, 36(04), 1033–1038. (in Chinese with English Abstract).

    Google Scholar 

  • Zhang, Y.Y., Sun, G.M., Ren, Q., & Zhao, D.Q. (2013) Foggy images classification based on features extraction and SVM. In 2013 International Conference on Software Engineering and Computer Science. https://doi.org/10.2991/icsecs-13.2013.30.

  • Zhao, B., Li, X. L., Lu, X. Q., & Wang, Z. G. (2018a). A CNN-RNN architecture for multi-label weather recognition. Neurocomputing, 322, 47–57. https://doi.org/10.1016/j.neucom.2018.09.048

    Article  Google Scholar 

  • Zhao, Q., Liu, S. C., & Zhang, B. X. (2018b). Multiactivation pooling method in convolutional neural networks for image recognition. Wireless Communications and Mobile Computing. https://doi.org/10.1155/2018/8196906

    Article  Google Scholar 

  • Zhuo, L., Jiang, L. Y., Zhu, Z. Q., Li, J. F., Zhang, J., & Long, H. X. (2017). Vehicle classification for large-scale traffic surveillance videos using convolutional neural networks. Machine Vision and Applications, 28(7), 793–802.

    Article  Google Scholar 

  • Zong, Q. H., Wang, Z. Y., Guo, Z. Y., & Ma, P. (2016). Remote sensing image classification based on wavelet transform and k-means algorithm. Journal of Hangzhou Normal University, 2, 203–207. (in Chinese with English Abstract).

    Google Scholar 

Download references

Acknowledgements

The authors are grateful to the editor and the anonymous reviewers for their meaningful suggestions and critical review of the manuscript. This study has been funded with support from the Major Science and Technology Innovation Projects of Shandong Province [No. 2019JZZY020103].

Author information

Authors and Affiliations

  1. Key Laboratory of Geomatics and Digital Technology of Shandong Province, Shandong University of Science and Technology, Qingdao, 266590, China

    Xiangwei Zhao, Kang Feng, Bo Wu, Jishan Luan & Min Ji

  2. Beijing Piesat Information Technology Co., Ltd, Beijing, 100195, China

    Jiaojiao Jiang

Authors
  1. Xiangwei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiaojiao Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kang Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bo Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jishan Luan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Min Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiangwei Zhao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, X., Jiang, J., Feng, K. et al. The Method of Classifying Fog Level of Outdoor Video Images Based on Convolutional Neural Networks. J Indian Soc Remote Sens 49, 2261–2271 (2021). https://doi.org/10.1007/s12524-021-01387-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12524-021-01387-6

Keywords

Search

Navigation