Skip to main content
SpringerLink
Log in

Unsupervised defect inspection algorithm based on cascaded GAN with edge repair feature fusion

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Surface defect inspection can greatly improve the efficiency of industrial production by replacing manual operations. However, in actual industrial scenarios, it is difficult to collect and manually label enough defect images. In addition, the complex backgrounds, diverse shapes and sizes, and broad random location distribution of defects in images make defect inspection more challenging. To address these issues, we propose an unsupervised defect inspection algorithm based on cascaded GAN (Generative Adversarial Networks) with edge repair feature fusion. In this algorithm, the edge repair network provides intact structural features for the defect repair network by means of a feature fusion method based on channel attention. For the edge repair network, we develop a deformable autoencoder, which fully utilizes the ability of deformable convolution to perceive very little contextual information to improve its ability to repair defect edges. Specifically, training requires only a few defect-free images and no labeled defect images. To verify the effectiveness of the proposed algorithm, we compare it with existing algorithms in terms of precision, the F1-measure, and the mIoU (mean Intersection over Union). The experimental results show that the proposed algorithm exhibits better defect inspection performance, especially for defects with rich forms and diverse positions against complex backgrounds.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Zheng X, Chen J, Wang H, et al (2020) A deep learning-based approach for the automated surface inspection of copper clad laminate images. Applied intelligence, 51(3):1-18. 10.1007%2Fs10489-020-01877-z

  2. Ren R, Hung T, Tan KC (2018) A generic deep-learning-based approach for automated surface inspection. IEEE Transactions on Cybernetics 48(3):929–940. https://doi.org/10.1109/TCYB.2017.2668395

    Article  Google Scholar 

  3. Annaby MH, Fouda YM, Rushdi MA (2019) Improved Normalized Cross-Correlation for Defect Detection in Printed-Circuit Boards. IEEE Transactions on Semiconductor Manufacturing, 32(2):199–211. https://doi.org/10.1109/TSM.2019.2911062

  4. Aiger D, Talbot H (2012) The phase only transform for unsupervised surface defect detection. Emerging topics in computer vision and its applications, pp 215-232. https://doi.org/10.1142/9789814343008_0011

  5. Zhou Q, Chen R, Huang B, Liu C, Yu J, Yu X (2019) An automatic surface defect inspection system for automobiles using machine vision methods. Sensors 19(3):644. https://doi.org/10.3390/s19030644

    Article  Google Scholar 

  6. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431-3440. https://doi.org/10.1109/CVPR.2015.7298965

  7. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. International conference on medical image computing and computer-assisted intervention, pp 234-241. https://doi.org/10.1007/978-3-319-24574-4_28

  8. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615

    Article  Google Scholar 

  9. Wang R, Guo Q, Lu S, Zhang C (2019) Tire defect detection using fully convolutional network. IEEE Access 7:43502–43510. https://doi.org/10.1109/ACCESS.2019.2908483

    Article  Google Scholar 

  10. Yu Z, Wu X, Gu X (2017) Fully convolutional networks for surface defect inspection in industrial environment. International conference on computer vision systems, pp 417-426. https://doi.org/10.1007/978-3-319-68345-4_37

  11. Dung CV (2019) Autonomous concrete crack detection using deep fully convolutional neural network. Autom Constr 99:52–58. https://doi.org/10.1016/j.autcon.2018.11.028

    Article  Google Scholar 

  12. Huang Y, Qiu C, Yuan K (2020) Surface defect saliency of magnetic tile. Vis Comput 36:85–96. https://doi.org/10.1007/s00371-018-1588-5

    Article  Google Scholar 

  13. Li S, Zhao X, Zhou G (2019) Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network. Computer-Aided Civil and Infrastructure Engineering 34(7):616–634. https://doi.org/10.1111/mice.12433

    Article  Google Scholar 

  14. Liu E, Chen K, Xiang Z, Zhang J (2020) Conductive particle detection via deep learning for ACF bonding in TFT-LCD manufacturing. J Intell Manuf 31(8):1037–1049. https://doi.org/10.1007/s10845-019-01494-9

    Article  Google Scholar 

  15. Dong Y, Wang J, Wang Z, Zhang X, Gao Y, Sui Q, Jiang P (2019) A deep-learning-based multiple defect detection method for tunnel lining damages. IEEE Access 7:182643–182657. https://doi.org/10.1109/ACCESS.2019.2931074

    Article  Google Scholar 

  16. Zou Q, Zhang Z, Li Q, Qi X, Wang Q, Wang S (2018) Deepcrack: learning hierarchical convolutional features for crack detection. IEEE Trans Image Process 28(3):1498–1512. https://doi.org/10.1109/TIP.2018.2878966

    Article  MathSciNet  Google Scholar 

  17. Tao X, Zhang D, Ma W, Liu X, Xu D (2018) Automatic metallic surface defect detection and recognition with convolutional neural networks. Applied Sciences-Basel 8(9):1575. https://doi.org/10.3390/app8091575

    Article  Google Scholar 

  18. Chen LC, Zhu Y, Papandreou G et al (2018) Encoder-decoder with Atrous separable convolution for semantic image segmentation. European Conference on Computer Vision, pp:833–851. https://doi.org/10.1007/978-3-030-01234-2_49

  19. Tabernik D, Ela S, Skvar J et al (2020) Segmentation-based deep-learning approach for surface-defect detection. J Intell Manuf 31(3):759–776. https://doi.org/10.1007/s10845-019-01476-x

    Article  Google Scholar 

  20. Li Y, Zhang D, Lee DJ (2019) Automatic fabric defect detection with a wide-and-compact network. Neurocomputing 329:329–338. https://doi.org/10.1016/j.neucom.2018.10.070

    Article  Google Scholar 

  21. Mei S, Yang H, Yin Z (2018) An unsupervised-learning-based approach for automated defect inspection on textured surfaces. IEEE Trans Instrum Meas 67(6):1266–1277. https://doi.org/10.1109/TIM.2018.2795178

    Article  Google Scholar 

  22. Haselmann M, Gruber D P, Tabatabai P (2018) Anomaly detection using deep learning based image completion. 2018 17th IEEE international conference on machine learning and applications, pp 1237-1242. https://doi.org/10.1109/ICMLA.2018.00201

  23. Youkachen S, Ruchanurucks M, Phatrapomnant T, et al (2019) Defect segmentation of hot-rolled steel strip surface by using convolutional auto-encoder and conventional image processing. 2019 10th international conference of information and communication Technology for Embedded Systems, pp 1-5. https://doi.org/10.1109/ICTEmSys.2019.8695928

  24. Kang G, Gao S, Yu L, Zhang D (2018) Deep architecture for high-speed railway insulator surface defect detection: Denoising autoencoder with multitask learning. IEEE Trans Instrum Meas 68(8):2679–2690. https://doi.org/10.1109/TIM.2018.2868490

    Article  Google Scholar 

  25. Zhao Z, Li B, Dong R, et al (2018) A surface defect detection method based on positive samples. Pacific Rim International Conference on Artificial Intelligence. Springer, Cham, pp. 473–481. https://doi.org/10.1007/978-3-319-97310-4_54

  26. Zhou F, Yang S, Fujita H, Chen D, Wen C (2020) Deep learning fault diagnosis method based on global optimization GAN for unbalanced data. Knowl-Based Syst 187:104837.1–104837.19. https://doi.org/10.1016/j.knosys.2019.07.008

    Article  Google Scholar 

  27. Yang H, Chen Y, Song K, Yin Z (2019) Multiscale feature-clustering-based fully convolutional autoencoder for fast accurate visual inspection of texture surface defects. IEEE Trans Autom Sci Eng 16(3):1450–1467. https://doi.org/10.1109/TASE.2018.2886031

    Article  Google Scholar 

  28. Chen Y, Zhang H, Liu L, Chen X, Zhang Q, Yang K, Xia R, Xie J (2020) Research on image inpainting algorithm of improved GAN based on two-discriminations networks. Appl Intell 51:3460–3474. https://doi.org/10.1007/s10489-020-01971-2

    Article  Google Scholar 

  29. Zhang J, Ding S, Zhang N, Shi Z (2016) Incremental extreme learning machine based on deep feature embedded. Int J Mach Learn Cybern 7(1):111–120. https://doi.org/10.1007/s13042-015-0419-5

    Article  Google Scholar 

  30. Zhang J, Ding S, Zhang N et al (2018) An overview on probability undirected graphs and their applications in image processing. Neurocomputing 321:156–168. https://doi.org/10.1016/j.neucom.2018.07.078

    Article  Google Scholar 

  31. Zhang J, Ding S, Zhang N, Xue Y (2016) Weight uncertainty in Boltzmann machine. Cogn Comput 8(6):1064–1073. https://doi.org/10.1007/s12559-016-9429-1

    Article  Google Scholar 

  32. Zhu X, Hu H, Lin S et al (2019) Deformable convnets v2: more deformable, better results. Proceedings of the IEEE conference on computer vision and pattern recognition:9308–9316. https://doi.org/10.1109/CVPR.2019.00953

  33. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. Proceedings of the IEEE conference on computer vision and pattern recognition:7132–7141. https://doi.org/10.1109/CVPR.2018.00745

Download references

Acknowledgements

This research work was supported in part by the National Natural Science Foundation of China (U1903213) and Xi’an Science and Technology Funds (GX2022).

Author information

Authors and Affiliations

  1. School of Information and Communications Engineering, Xi’an Jiaotong University, Xi’an, 710049, China

    Lijun He, Nan Shi, Kainnat Malik & Fan Li

Authors
  1. Lijun He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nan Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kainnat Malik
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fan Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, L., Shi, N., Malik, K. et al. Unsupervised defect inspection algorithm based on cascaded GAN with edge repair feature fusion. Appl Intell 52, 2051–2069 (2022). https://doi.org/10.1007/s10489-021-02556-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02556-3

Keywords

Search

Navigation