Skip to main content
SpringerLink
Log in

Scale–Space Data Augmentation for Deep Transfer Learning of Crack Damage from Small Sized Datasets

  • Published:
Journal of Nondestructive Evaluation Aims and scope Submit manuscript

Abstract

Crack damage is commonly observed for civil structures and infrastructure in service. The recent years have witnessed an excessive utilization of deep learning models for realizing autonomous and machine-vision based crack detection. Given this trend, this paper recognizes two entwined challenges: the preparation of large-scale training data and the detection of simple crack damage amid complex scenes. To address them, a novel data augmentation technique is proposed considering crack characteristics in images for realizing deep transfer learning using very small datasets. Numerical experimentation is conducted based on two types of crack datasets (concrete structures and asphalt pavement), each of which has only tens of images containing complex scenes. When evaluating the performance, a sliding-window based rating scheme is proposed, which is much stricter than the conventional bounding-box based approach. Quantitative performance analysis shows the acceptable performance (e.g., an overall accuracy of 93.81%, an F-2 score of 74.4%, and a very high recall of 91% for the crack detection in concrete images). The result demonstrates the effectiveness of the proposed data augmentation method and the superior transferability if the transfer learning is carried out through a fully fine-tuned training process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Chien, C., Martin, W., Meyer, A., Aggarwal, J.: Detection of cracks on highway pavements. Interim Report Texas Univ Austin Center for Transportation Research (1983)

  2. Haas, C., Hendrickson, C.: Computer-based model of pavement surfaces. Transp. Res. Rec. 1260, 91–98 (1990)

    Google Scholar 

  3. Ritchie, S.G.: Digital imaging concepts and applications in pavement management. J. Transp. Eng. 116(3), 287–298 (1990)

    Article  Google Scholar 

  4. Abdel-Qader, I., Kelly, M.: Analysis of edge-detection techniques for crack identification in bridges. ASCE J. Comput. Civil Eng. 17(4), 255–263 (2003)

    Article  Google Scholar 

  5. Hutchinson, T.C., Chen, Z.: Improved image analysis for evaluating concrete damage. J. Comput. Civil Eng. 20, 210–216 (2006)

    Article  Google Scholar 

  6. Nishikawa, T., Yoshida, J., Sugiyama, T., Fujino, Y.: Concrete crack detection by multiple sequential image filtering. Comput.-Aided Civil Infrastruct. Eng. 27(1), 29–47 (2012)

    Article  Google Scholar 

  7. Fujita, Y., Hamamoto, Y.: A robust automatic crack detection method from noisy concrete surfaces. Mach. Vis. Appl. 22(2), 245–254 (2011)

    Article  Google Scholar 

  8. Lindeberg, T.: Edge detection and ridge detection with automatic scale selection. Int. J. Comput. Vision 30(2), 117–156 (1998)

    Article  Google Scholar 

  9. Chen, Z., Hutchinson, T.C.: Image-based framework for concrete surface crack monitoring and quantification. Adv. Civil Eng. (2010). https://doi.org/10.1155/2010/215295

    Article  Google Scholar 

  10. Liu, Z., Suandi, S.A., Ohashi, T., Ejima, T.: Tunnel crack detection and classification system based on image processing. In: Machine Vision Applications in Industrial Inspection X. International Society for Optics and Photonics, pp. 145–152 (2002)

  11. Mohan, A., Poobal, S.: Crack detection using image processing: a critical review and analysis. Alexandria Eng. J. 57(2), 787–798 (2018)

    Article  Google Scholar 

  12. Chen, Z., Derakhshani, R., Halmen, C., Kevern, J.T.: A texture-based method for classifying cracked concrete surfaces from digital images using neural networks. In: The 2011 International Joint Conference on Neural Networks (IJCNN). IEEE, pp. 2632–2637 (2011)

  13. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp. 248–255 (2009)

  14. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)

    Article  Google Scholar 

  15. Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in neural information processing systems, pp. 3320–3328 (2014)

  16. Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. arXiv preprint arXiv:14053531 (2014)

  17. Wang, J., Perez, L.: The effectiveness of data augmentation in image classification using deep learning. Convolutional Neural Netw. Vis. Recognit. (2017)

  18. Zhang, L., Yang, F., Zhang, Y.D., Zhu, Y.J.: Road crack detection using deep convolutional neural network. In: 2016 IEEE International Conference on Image Processing (ICIP). IEEE, pp. 3708–3712 (2016)

  19. Cha, Y.-J., Choi, W., Büyüköztürk, O.: Deep learning-based crack damage detection using convolutional neural networks. Comput.-Aided Civil Infrastruct. Eng. 32(5), 361–378 (2017). https://doi.org/10.1111/mice.12263

    Article  Google Scholar 

  20. Cha, Y.J., Choi, W., Suh, G., Mahmoudkhani, S., Büyüköztürk, O.: Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput.-Aided Civil Infrastruct. Eng. 33(9), 731–747 (2018)

    Article  Google Scholar 

  21. Beckman, G.H., Polyzois, D., Cha, Y.-J.: Deep learning-based automatic volumetric damage quantification using depth camera. Autom. Constr. 99, 114–124 (2019). https://doi.org/10.1016/j.autcon.2018.12.006

    Article  Google Scholar 

  22. Ali, R., Cha, Y.-J.: Subsurface damage detection of a steel bridge using deep learning and uncooled micro-bolometer. Constr. Build. Mater. 226, 376–387 (2019)

    Article  Google Scholar 

  23. Dung, C.V.: Autonomous concrete crack detection using deep fully convolutional neural network. Autom. Constr. 99, 52–58 (2019)

    Article  Google Scholar 

  24. Liu, Y., Yao, J., Lu, X., Xie, R., Li, L.: DeepCrack: a deep hierarchical feature learning architecture for crack segmentation. Neurocomputing 338, 139–153 (2019)

    Article  Google Scholar 

  25. Ni, F., Zhang, J., Chen, Z.: Pixel-level crack delineation in images with convolutional feature fusion. Struct. Control Health Monit. 26(1), e2286 (2019)

    Article  Google Scholar 

  26. Ni, F., Zhang, J., Chen, Z.: Zernike-moment measurement of thin-crack width in images enabled by dual-scale deep learning. Comput.-Aided Civil Infrastruct. Eng. 34(5), 367–384 (2019)

    Article  Google Scholar 

  27. Gao, Y., Mosalam, K.M.: Deep transfer learning for image-based structural damage recognition. Comput.-Aided Civil Infrastruct. Eng. 33(9), 748–768 (2018)

    Article  Google Scholar 

  28. Choi, W., Cha, Y.: SDDNet: real-time crack segmentation. IEEE Trans. Industr. Electron. 67(9), 8016–8025 (2020). https://doi.org/10.1109/TIE.2019.2945265

    Article  Google Scholar 

  29. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)

  30. Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)

  31. Fan, Q., Brown, L., Smith, J.: A closer look at faster R-CNN for vehicle detection. In: 2016 IEEE Intelligent Vehicles Symposium (IV). IEEE, pp. 124–129 (2016)

  32. Jiang, H.: Learned-Miller E Face detection with the faster R-CNN. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). IEEE, pp. 650–657 (2017)

  33. Xu, Y., Yu, G., Wang, Y., Wu, X., Ma, Y.: Car detection from low-altitude UAV imagery with the faster R-CNN. J. Adv. Transp. (2017)

  34. Zhang, L., Lin, L., Liang, X., He, K.: Is faster r-cnn doing well for pedestrian detection? In: European Conference on Computer Vision, pp. 443–457. Springer (2016)

  35. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. arXiv e-prints: arXiv:1311.2901 (2013)

  36. Chen, Z.: Small crack-damage images with masks (2020)

  37. Lindeberg, T.: Scale-Space Theory in Computer Vision, vol. 256. Springer, New York (2013)

    MATH  Google Scholar 

  38. Lowe, D.: Distinctive image features fromscale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  39. Lindeberg, T.: Detecting salient blob-like image structures and their scales with a scale-space primal sketch: a method for focus-of-attention. Int. J. Comput. Vis. 11(3), 283–318 (1993)

    Article  Google Scholar 

  40. Harris, C.G., Stephens, M.A.: combined corner and edge detector. In: Alvey Vision Conference. vol 50. Citeseer, pp. 10–5244 (1988)

  41. Mikolajczyk, K., Schmid, C.: Scale & affine invariant interest point detectors. Int. J. Comput. Vis. 60(1), 63–86 (2004)

    Article  Google Scholar 

  42. Chen, Z.: Identifying Structural Damage from Images. UC San Diego, San Diego (2009)

    Google Scholar 

  43. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia. ACM, pp. 675–678 (2014)

  44. Saito, T., Rehmsmeier, M.: The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE 10(3), e0118432 (2015)

    Article  Google Scholar 

  45. Chinchor, N.: The statistical significance of the MUC-4 results. In: Proceedings of the 4th Conference on Message Understanding. Association for Computational Linguistics, pp. 30–50 (1992)

  46. Davis, J., Goadrich, M.: The relationship between precision-recall and ROC curves. In: Proceedings of the 23rd International Conference on Machine Learning. ACM, pp. 233–240 (2006)

Download references

Acknowledgements

This material is partially based upon work supported by the National Science Foundation (NSF) under Award Number IIA-1355406. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NSF.

Author information

Authors and Affiliations

  1. Department of Computer Science Electrical Engineering, University of Missouri-Kansas City, Kansas City, USA

    Shimin Tang

  2. Department of Civil and Mechanical Engineering, University of Missouri-Kansas City, Kansas City, USA

    ZhiQiang Chen

Authors
  1. Shimin Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. ZhiQiang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to ZhiQiang Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, S., Chen, Z. Scale–Space Data Augmentation for Deep Transfer Learning of Crack Damage from Small Sized Datasets. J Nondestruct Eval 39, 70 (2020). https://doi.org/10.1007/s10921-020-00715-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10921-020-00715-z

Keywords

Search

Navigation