Skip to main content
SpringerLink
Log in

TAFFNet: Two-Stage Attention-Based Feature Fusion Network for Surface Defect Detection

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

It is important to detect surface defects for controlling product quality and prolonging equipment life. However, detecting surface defects quickly and accurately is still a great challenge due to the complexity of the environment and surface defects. Aiming at the issue, this paper proposes a two-stage attention-based feature fusion network (TAFFNet) to make full use of each level feature for surface defect segmentation. Specifically, the network uses Resnet50 as the backbone network to obtain features, and then extracts multi-scale feature by the atrous convolution feature extraction module. In order to make the features at all levels contain more defect information, the attention-based adjacent feature fusion module is applied to fuse features with adjacent layers; then use the attention-based high-level feature fusion module to merge features with all upper layers, so all level features not only contain multi-scale context but also obtain more defect details. Finally, all features are cascaded together to achieve accurate segmentation of surface defects. In addition, TAFFNet uses a hybrid loss function to overcome blurry boundaries. The experimental results on three surface defects datasets (SD900, MT, and CFD) show that the proposed network outperforms other 13 methods in terms of PR curve, F1, MAE, and mIoU.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7

Similar content being viewed by others

References

  1. Song, G., Song, K., & Yan, Y. (2020). EDRNet: Encoder-decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement, 69(12), 9709–9719.

    Article  Google Scholar 

  2. Zhou, X., Wang, Y., Zhu, Q., Mao, J., Xiao, C., Lu, X., & Zhang, H. (2020). A surface defect detection framework for glass bottle bottom using visual attention model and wavelet transform. IEEE Transactions on Industrial Informatics, 16(4), 2189–2201.

    Article  Google Scholar 

  3. Su, B., Chen, H., Chen, P., Bian, G., Liu, K., & Liu, W. (2021). Deep learning-based solar-cell manufacturing defect detection with complementary attention network. IEEE Transactions on Industrial Informatics, 17(6), 4084–4095.

    Article  Google Scholar 

  4. Tang, Y., Zhang, A. A., Luo, L., Wang, G., & Yang, E. (2021). Pixel-level pavement crack segmentation with encoder-decoder network. Measurement, 184, 109914.

    Article  Google Scholar 

  5. Nieniewski, M. (2020). Morphological detection and extraction of rail surface defects. IEEE Transactions on Instrumentation and Measurement, 69(9), 6870–6879.

    Article  Google Scholar 

  6. Krummenacher, G., Ong, C. S., Koller, S., Kobayashi, S., & Buhmann, J. M. (2018). Wheel defect detection with machine learning. IEEE Transactions on Intelligent Transportation Systems, 19(4), 1176–1187.

    Article  Google Scholar 

  7. Yapi, D., Allili, M. S., & Baaziz, N. (2018). Automatic fabric defect detection using learning-based local textural distributions in the contourlet domain. IEEE Transactions on Automation Science and Engineering, 15(3), 1014–1026.

    Article  Google Scholar 

  8. Li, W., & Tsai, D. (2011). Defect inspection in low-contrast LCD images using hough transform-based nonstationary line detection. IEEE Transactions on Industrial Informatics, 7(1), 136–147.

    Article  MathSciNet  Google Scholar 

  9. Eren, L., Ince, T., & Kiranyaz, S. (2019). A generic intelligent bearing fault diagnosis system using compact adaptive 1D CNN classifier. Journal of Signal Processing Systems, 91(2), 179–189.

    Article  Google Scholar 

  10. Rathore, M. S., & Harsha, S. P. (2022). Degradation pattern of high speed roller bearings using a data-driven deep learning approach. Journal of Signal Processing Systems. https://doi.org/10.1007/s11265-022-01761-8

    Article  Google Scholar 

  11. Yuan, Y., Virupakshappa, K., & Oruklu, E. (2022). FPGA implementation of an ultrasonic flaw detection algorithm based on convolutional neural networks. Journal of Signal Processing Systems. https://doi.org/10.1007/s11265-022-01756-5

    Article  Google Scholar 

  12. Borji, A., Frintrop, S., Sihite, D. N., & Itti, L. (2012). Adaptive object tracking by learning background context. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 23–30.

  13. Abdulmunem, A., Lai, Y.-K., & Sun, X. (2016). Saliency guided local and global descriptors for effective action recognition. Computational Visual Media, 2(1), 97–106.

    Article  Google Scholar 

  14. Zhu, L., Chen, J., Hu, X., Fu, C., Xu, X., Qin, J., & Heng, P. (2020). Aggregating attentional dilated features for salient object detection. IEEE Transactions on Circuits and Systems for Video Technology, 30(10), 3358–3371.

    Article  Google Scholar 

  15. Pang, Y., Zhao, X., Zhang, L., & Lu, H. (2020). Multi-scale interactive network for salient object detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 9410–9419.

  16. Wang, B., Chen, Q., Zhou, M., Zhang, Z., & Gai, K. (2020). Progressive feature polishing network for salient object detection. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pp 12128–12135.

  17. Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254–1259.

    Article  Google Scholar 

  18. Harel, J., Koch C., & Perona P. (2006). Graph-based visual saliency. In Proceedings Neural Information Processing Systems (NIPS), pp 545–552.

  19. Hou, X., & Zhang, L. (2007). Saliency Detection: A Spectral Residual Approach. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 1–8.

  20. Guo, C., Ma, Q., & Zhang, L.(2008). Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 1–8.

  21. Achanta, R., Hemami, S., Estrada, F., & Susstrunk, S. (2009). Frequency-tuned salient region detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 1597–1604.

  22. Cheng, M., Zhang, G., Mitra, N. J., Huang, X., & Hu, S. (2011). Global contrast based salient region detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 409–416.

  23. Goferman, S., Zelnik-Manor, L., & Tal, A. (2012). Context-Aware Saliency Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(10), 1915–1926.

    Article  Google Scholar 

  24. Xie, Y., Lu, H., & Yang, M. (2013). Bayesian saliency via low and mid level cues. IEEE Transactions on Image Processing, 22(5), 1689–1698.

    Article  MathSciNet  MATH  Google Scholar 

  25. Du, S., & Chen, S. (2014). Salient Object Detection via Random Forest. IEEE Signal Processing Letters, 21(1), 51–54.

    Article  Google Scholar 

  26. Peng, H., Li, B., Ling, H., Hu, W., Xiong, W., & Maybank, S. J. (2017). Salient object detection via structured matrix decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 818–832.

    Article  Google Scholar 

  27. Tang, C., Wang, P., Zhang, C., & Li, W. (2017). Salient object detection via weighted low rank Matrix recovery. IEEE Signal Processing Letters, 24(4), 490–494.

    Article  Google Scholar 

  28. Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 3431–3440.

  29. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Proceedings Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp 234–241.

  30. Hou, Q., Cheng, M., Hu, X., Borji, A., Tu, Z., & Torr, P. (2017). Deeply supervised salient object detection with short connections. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 5300–5309.

  31. Zhang P., Wang, D., Lu, H., Wang, H., & Ruan, X. (2017). Amulet: Aggregating multi-level convolutional features for salient object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 202–211.

  32. Zhang, X., Wang, T., Qi, J., Lu, H., & Wang, G. (2018) Progressive attention guided recurrent network for salient object detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 714–722.

  33. Wu, Z., Su, L., & Huang, Q. (2019). Cascaded partial decoder for fast and accurate salient object detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 3902–3911.

  34. Chen, S., Tan, X., Wang, B., & Hu, X. (2018). Reverse attention for salient object detection. In Proceedings of the European Conference on Computer Vision (ECCV), pp 236–252.

  35. Liu, J., Hou, Q., Cheng, M., Feng, J., & Jiang, J. (2019). A simple pooling-based design for real-time salient object detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 3912–3921.

  36. Wei, J., Wang, S., & Huang, Q. (2020). F3Net: fusion, feedback and focus for salient object detection. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pp 12321–12328.

  37. Chen, Z., Xu, Q., Cong, R., & Huang, Q. (2020). Global context-aware progressive aggregation network for salient object detection. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pp 10599–10606.

  38. Qin, X., Zhang, Z., Huang, C., Gao, C., Dehghan, M., & Jagersand, M. (2019). BASNet: Boundary-aware salient object detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 7471–7481.

  39. Deng, Z., Hu, X., Zhu, L., Xu, X., Qin, J., Han, G., & Heng, P. (2018). R3net: Recurrent residual refinement network for saliency detection. In Proceedings International Joint Conference on Artificial Intelligence (IJCAI), pp 684–690.

  40. Zhao, J., Liu, J., Fan, D., Cao, Y., Yang, J., & Cheng, M. (2019). EGNet: Edge guidance network for salient object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp 8778–8787.

  41. Woo, S., Park J., Lee, J., & Kweon, I. S. (2018). CBAM: Convolutional block attention module. In Proceedings of the IEEE International Conference on Computer Vision (ECCV), pp 3–19.

  42. Peng, C., Zhang, X., Yu, G., Luo, G., & Sun, J. (2017). Large kernel matters improve semantic segmentation by global convolutional network. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 1743–1751.

  43. Song, G., Song, K., & Yan, Y. (2020). Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering, 128, 106000.

    Article  Google Scholar 

  44. Huang, Y., Qiu, C., & Yuan, K. (2020). Surface defect saliency of magnetic tile. The Visual Computer, 36(1), 85–96.

    Article  MathSciNet  Google Scholar 

  45. Shi, Y., Cui, L., Qi, Z., Meng, F., & Chen, Z. (2016). Automatic road crack detection using random structured forests. IEEE Transactions on Intelligent Transportation Systems, 17(12), 3434–3445.

    Article  Google Scholar 

  46. Qin, X., Zhang, Z., Huang, C., Dehghan, M., & Jagersand, M. (2020). U2-net: Going deeper with nested u-structure for salient object detection. Pattern Recognition, 106, 107404.

    Article  Google Scholar 

Download references

Funding

This work is supported by the Fundamental Research Funds for the Central Universities of China (2021MS092).

Author information

Authors and Affiliations

  1. School of Control and Computer Engineering, North China Electric Power University, Baoding, 071003, China

    Jingang Cao

  2. School of Control and Computer Engineering, North China Electric Power University, Beijing, 102206, China

    Guotian Yang & Xiyun Yang

Authors
  1. Jingang Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guotian Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiyun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jingang Cao.

Ethics declarations

Conflicts of Interest

We declare that we have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, J., Yang, G. & Yang, X. TAFFNet: Two-Stage Attention-Based Feature Fusion Network for Surface Defect Detection. J Sign Process Syst 94, 1531–1544 (2022). https://doi.org/10.1007/s11265-022-01801-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-022-01801-3

Keywords

Search

Navigation