Skip to main content

Advertisement

SpringerLink
Log in

A steel surface defect inspection approach towards smart industrial monitoring

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

With the advance in Industry 4.0, smart industrial monitoring has been proposed to timely discover faults and defects in industrial processes. Steel is widely used in manufacturing equipment, and steel surface defect inspection is of great significance to the normal operation of steel equipment in manufacturing workshops. In steel defect inspection systems, industrial inspection robots generate images via scanning steel surface, and processors perform surface defect inspection algorithms on images. We focus on applying advanced object detection techniques to surface defect inspection algorithm for sheet steel. In the proposed steel surface defect inspection model, a deformable convolution enhanced backbone network firstly extracts complex features from multi-shape steel surface defects. Then the feature fusion network with balanced feature pyramid generates high-quality multi-resolution feature maps for the inspection of multi-size defects. Finally, detector network achieves the localization and classification of steel surface defects. The proposed model is evaluated on a typical steel surface defect dataset. Our model achieves 0.805 mAP, 0.144 higher than baseline models, and our model shows high efficiency in inference. Experiments are performed to reveal the effect of employed approaches, and results also show our model achieves a balance between inspection performance and inference efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Badmos, O., Kopp, A., Bernthaler, T., & Schneider, G. (2020). Image-based defect detection in lithium-ion battery electrode using convolutional neural networks. Journal of Intelligent Manufacturing, 31, 885–897.

    Article  Google Scholar 

  • Bustillo, A., Pimenov, D. Y., Matuszewski, M., & Mikolajczyk, T. (2018). Using artificial intelligence models for the prediction of surface wear based on surface isotropy levels. Robotics and Computer-integrated Manufacturing, 53, 215–227.

    Article  Google Scholar 

  • Chu, M., Gong, R., Gao, S., & Zhao, J. (2017). Steel surface defects recognition based on multi-type statistical features and enhanced twin support vector machine. Chemometrics and Intelligent Laboratory Systems, 171, 140–150.

    Article  Google Scholar 

  • Cohen, Y., Faccio, M., Pilati, F., & Yao, X. (2019). Design and management of digital manufacturing and assembly systems in the industry 4.0 era. The International Journal of Advanced Manufacturing Technology, 105(9), 3565–3577.

    Article  Google Scholar 

  • Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., Wei, Y. (2017). Deformable convolutional networks. In: International Conference on Computer Vision, pp. 764–773.

  • de Vooys, A., & van der Weijde, H. (2011). Investigating cracks and crazes on coated steel with simultaneous svet and eis. Progress in Organic Coatings, 71(3), 250–255.

    Article  Google Scholar 

  • Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010). The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88(2), 303–338.

    Article  Google Scholar 

  • Ghorai, S., Mukherjee, A., Gangadaran, M., & Dutta, P. K. (2013). Automatic defect detection on hot-rolled flat steel products. IEEE Transactions on Instrumentation and Measurement, 62(3), 612–621.

    Article  Google Scholar 

  • Girshick, R. (2015). Fast r-cnn. In: International Conference on Computer Vision, pp. 1440–1448.

  • Girshick, R., Donahue, J., Darrell, T., Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In: Conference on Computer Vision and Pattern Recognition, pp. 580–587.

  • He, D., Xu, K., & Zhou, P. (2019). Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering, 128, 290–297.

    Article  Google Scholar 

  • He, K., Gkioxari, G., Dollar, P., Girshick, R. (2017). Mask r-cnn. In: International Conference on Computer Vision, pp. 2980–2988.

  • He, K., Zhang, X., Ren, S., Sun, J. (2014). Spatial pyramid pooling in deep convolutional networks for visual recognition. In: European Conference on Computer Vision, pp. 346–361.

  • He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep residual learning for image recognition. In: Conference on Computer Vision and Pattern Recognition, pp. 770–778.

  • He, Y., Song, K., Meng, Q., & Yan, Y. (2020). An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Transactions on Instrumentation and Measurement, 69(4), 1493–1504.

    Article  Google Scholar 

  • Hu, X., Xu, X., Xiao, Y., Chen, H., He, S., Qin, J., et al. (2019). Sinet: a scale-insensitive convolutional neural network for fast vehicle detection. IEEE Transactions on Intelligent Transportation Systems, 20(3), 1010–1019.

    Article  Google Scholar 

  • Khan, S. (2018). Ant colony optimization (aco) based data hiding in image complex region. International Journal of Electrical and Computer Engineering, 8(1), 379–389.

    Google Scholar 

  • Leinenbach, C., Koster, M., & Schindler, H. (2012). Fatigue assessment of defect-free and defect-containing brazed steel joints. Journal of Materials Engineering and Performance, 21(5), 739–747.

    Article  Google Scholar 

  • Li, W., Li, H., Wu, Q., Chen, X., & Ngan, K. N. (2019). Simultaneously detecting and counting dense vehicles from drone images. IEEE Transactions on Industrial Electronics, 66(12), 9651–9662.

    Article  Google Scholar 

  • Lin, T., Dollar, P., Girshick, R., He, K., Hariharan, B., Belongie, S. (2017). Feature pyramid networks for object detection. In: Conference on Computer Vision and Pattern Recognition, pp. 936–944.

  • Liu, K., Wang, H., Chen, H., Qu, E., Tian, Y., & Sun, H. (2017). Steel surface defect detection using a new haar-weibull-variance model in unsupervised manner. IEEE Transactions on Instrumentation and Measurement, 66(10), 2585–2596.

    Article  Google Scholar 

  • Luo, Q., Sun, Y., Li, P., Simpson, O., Tian, L., & He, Y. (2019). Generalized completed local binary patterns for time-efficient steel surface defect classification. IEEE Transactions on Instrumentation and Measurement, 68(3), 667–679.

    Article  Google Scholar 

  • Lv, X., Duan, F., Jiang, J., Fu, X., & Gan, L. (2020). Deep active learning for surface defect detection. Sensors, 20(6), 1650.

    Article  Google Scholar 

  • Nioi, M., Pinna, C., Celotto, S., Swart, E., Farrugia, D., Husain, Z., et al. (2019). Finite element modelling of surface defect evolution during hot rolling of silicon steel. Journal of Materials Processing Technology, 268, 181–191.

    Article  Google Scholar 

  • Pang, J., Chen, K., Shi, J., Feng, H., Ouyang, W., Lin, D. (2019). Libra r-cnn: towards balanced learning for object detection. In: Conference on Computer Vision and Pattern Recognition, pp. 821–830.

  • Paulraj, M.P., Shukry, A.M.M., Yaacob, S., Adom, A.H., Krishnan, R.P. (2010). Structural steel plate damage detection using dft spectral energy and artificial neural network. In: International Colloquium on Signal Processing and Its Applications, pp. 1–6.

  • Pimenov, D. Y., Bustillo, A., & Mikolajczyk, T. (2018). Artificial intelligence for automatic prediction of required surface roughness by monitoring wear on face mill teeth. Journal of Intelligent Manufacturing, 29(5), 1045–1061.

    Article  Google Scholar 

  • Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster r-cnn: towards real-time object detection with region proposal networks. Conference and Workshop on Neural Information Processing Systems, 2015, 91–99.

    Google Scholar 

  • Simonyan, K., Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations.

  • Song, K., & Yan, Y. (2013). A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science, 285, 858–864.

    Article  Google Scholar 

  • Sun, J., Li, C., Wu, X., Palade, V., & Fang, W. (2019). An effective method of weld defect detection and classification based on machine vision. IEEE Transactions on Industrial Informatics, 15(12), 6322–6333.

    Article  Google Scholar 

  • Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A. (2015). Going deeper with convolutions. In: Conference on Computer Vision and Pattern Recognition, pp. 1–9.

  • Tabernik, D., Sela, S., Skvarc, J., & Skocaj, D. (2020). Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing, 31, 759–776.

    Article  Google Scholar 

  • Uijlings, J., Sande, K. E., Gevers, T., & Smeulders, A. W. M. (2013). Selective search for object recognition. International Journal of Computer Vision, 104(2), 154–171.

    Article  Google Scholar 

  • Vithanage, R. K. W., Harrison, C. S., & De Silva, A. K. M. (2019). Autonomous rolling-stock coupler inspection using industrial robots. Robotics and Computer-integrated Manufacturing, 59, 82–91.

    Article  Google Scholar 

  • Yun, J. P., Choi, D., Jeon, Y., Park, C., & Kim, S. W. (2014). Defect inspection system for steel wire rods produced by hot rolling process. The International Journal of Advanced Manufacturing Technology, 70(9), 1625–1634.

    Article  Google Scholar 

  • Yun, J. P., Choi, S., Kim, J., & Kim, S. W. (2009). Automatic detection of cracks in raw steel block using gabor filter optimized by univariate dynamic encoding algorithm for searches (udeas). Ndt & E International, 42(5), 389–397.

    Article  Google Scholar 

  • Zhang, X., Kano, M., Tani, M., Mori, J., Ise, J., & Harada, K. (2020). Prediction and causal analysis of defects in steel products: Handling nonnegative and highly overdispersed count data. Control Engineering Practice, 95, 104,258.

    Article  Google Scholar 

  • Zhang, J., Li, H., Yang, B., Wu, B., & Zhu, S. (2020). Fatigue properties and fatigue strength evaluation of railway axle steel: Effect of micro-shot peening and artificial defect. International Journal of Fatigue, 132, 105,379.

    Article  Google Scholar 

  • Zhang, C., Xie, Y., Liu, D., & Wang, L. (2017). Fast threshold image segmentation based on 2d fuzzy fisher and random local optimized qpso. IEEE Transactions on Image Processing, 26(3), 1355–1362.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by a grant from the Institute for Guo Qiang, Tsinghua University (No. 2019GQG0002), and supported by National Natural Science Foundation of China (No. 41876098).

Author information

Authors and Affiliations

  1. Department of Automation, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, 100084, China

    Ruiyang Hao, Bingyu Lu & Biqing Huang

  2. School of Automation Science and Electrical Engineering, Beihang University, Beijing, 100191, China

    Ying Cheng

  3. Division of Information Science, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China

    Xiu Li

Authors
  1. Ruiyang Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bingyu Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Biqing Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xiu Li or Biqing Huang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hao, R., Lu, B., Cheng, Y. et al. A steel surface defect inspection approach towards smart industrial monitoring. J Intell Manuf 32, 1833–1843 (2021). https://doi.org/10.1007/s10845-020-01670-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-020-01670-2

Keywords

Search

Navigation