Skip to main content
SpringerLink
Log in

A novel CNN structure for fine-grained classification of Chinese calligraphy styles

  • Original Paper
  • Published:
International Journal on Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

Chinese calligraphy is a valuable cultural heritage belonging to the world. It is liked by many people, and our mission is to endeavor to pursue calligraphy and make contributions to the business with technical means. The automatic recognition of the styles of calligraphy by image processing techniques has important meaning in arts collection and auction, etc. Traditional feature operators have some drawbacks that leave room for modern methods like convolutional neural network (CNN). However, most of the studies focus on the classification of five basic fonts that is somewhat different from styles. In this paper, four kinds of styles belonging to standard font are classified with a novel CNN structure where two squeeze-and-excitation modules that emphasize informative feature maps and suppress useless features are embedded after convolution layers, and a Haar transform layer that fuses the features is imposed before softmax layer. Experiment result shows the significance of the proposed structure over other networks in both font and style classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. https://github.com/tensorflow/models/tree/master/research/slim#Pretrained.

References

  1. Yao, C., Cheng, G.: Approximative Bayes optimality linear discriminant analysis for Chinese handwriting character recognition. Neurocomputing 207, 346–353 (2016)

    Article  Google Scholar 

  2. Yang, W., Jin, L., Xie, Z., et al.: Improved deep convolutional neural network for online handwritten Chinese character recognition using domain-specific knowledge. In: 2015 13th international conference on document analysis and recognition (ICDAR). IEEE, pp. 551–555 (2015)

  3. Gao, P., Wu, J., Yuan, L., et al.: Fast Chinese calligraphic character recognition with large-scale data. Multimed. Tools Appl. 74(17), 7221–7238 (2015)

    Article  Google Scholar 

  4. Gao, P., Wu, J., Yuan, L., et al.: Fast Chinese calligraphic character recognition with large-scale data. Multimed. Tools Appl. 74(17), 7221–7238 (2015)

    Article  Google Scholar 

  5. Tao, D., Lin, X., Jin, L., et al.: Principal component 2-D long short-term memory for font recognition on single Chinese characters. IEEE Trans. Cybern. 46(3), 756 (2016)

    Article  Google Scholar 

  6. Zhang, X., Nagy, G.: Style comparisons in calligraphy. Doc. Recognit. Retr. XIX 8297(2), 263–271 (2012)

    Google Scholar 

  7. Zhang, Y., Liu, Y., He, J., et al.: Recognition of calligraphy style based on global feature defontor. In: IEEE International Conference on Multimedia and Expo. IEEE, pp. 1–6 (2013)

  8. Gao, P., Wu, J., Lin, Y., et al.: Fast image-based chinese calligraphic character retrieval on large scale data. In: Acm/ieee-Cs Joint Conference on Digital Libraries. IEEE Press, pp. 211–218 (2014)

  9. Shi, Z., Xu, B., Zheng, X., et al.: A Chinese character structure preserved denoising method for Chinese tablet calligraphy document images based on KSVD dictionary learning. Multimed. Tools Appl. 76(13), 1–16 (2016)

    Google Scholar 

  10. Imdad, A., Bres, S., Eglin, V., et al.: Writer identification using steered hermite features and SVM. In: International Conference on Document Analysis and Recognition. IEEE, pp. 839–843 (2017)

  11. Vásquez, J.L., Travieso, C.M., Alonso, J.B.: Using calligraphies features for off line writer identification. In: International Carnahan Conference on Security Technology. IEEE, pp. 1–6 (2014)

  12. Sun, R., Lian, Z., Tang, Y., et al.: Aesthetic visual quality evaluation of Chinese handwritings. In: International Conference on Artificial Intelligence. AAAI Press, pp. 2510–2516 (2015)

  13. Hou, S., Xu, P.: An algorithm of calligraphy beautification based on improved velocity and width model. In: International Conference on Computer Science. IEEE, pp. 124–127 (2015)

  14. Liu, L., Xia, W., Jin, L., et al.: A Kai style contour beautification method for Chinese handwriting characters. In: IEEE International Conference on Systems Man and Cybernetics. IEEE, pp. 3644–3649 (2010)

  15. Xiao, J., Xiao, J., Xiao, J.: Automatic generation of large-scale handwriting fonts via style learning. In: SIGGRAPH ASIA 2016 Technical Briefs. ACM, p. 12 (2016)

  16. Zhang, S.L., Xie, Z.K., Qi, Y.: Transformation of Chinese calligraphy style based on feature matrix editing. In; International Conference on Computer Science and Network Technology. IEEE, pp. 159–162 (2016)

  17. Jie, C., Ma, Y.H.: Novel Chinese calligraphy style generation based on curve analogy. International Workshop on Intelligent Systems and Applications. IEEE, pp. 1–3 (2011)

  18. Zhuang, Y., Lu, W., Wu, J.: Latent Style Model: Discovering Writing Styles for Calligraphy Works. Academic Press, Inc., Cambridge (2009)

    Google Scholar 

  19. Huang, S., Zhong, Z., Jin, L., et al.: DropRegion training of inception font network for high-performance Chinese font recognition[J]. Pattern Recogn. 77, 395–411 (2018)

    Article  Google Scholar 

  20. Wang, X., Girshick, R., Gupta, A., et al.: Non-local Neural Networks, p. 10 (2017). arXiv preprint arXiv:1711.07971

  21. Hu, J., Shen, L., Sun, G.: Squeeze-and-Excitation Networks, p. 7 (2017). arXiv preprint arXiv:1709.01507

  22. Szegedy, C., Ioffe, S., Vanhoucke, V. et al: Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI, vol. 4, no 12 (2017)

  23. Huang, G., Liu, Z., Van Der Maaten, L., et al.: Densely connected convolutional networks[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700–4708 (2017)

  24. Xie, S., Girshick, R., Dollár, P., et al.: Aggregated residual transformations for deep neural networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp. 5987–5995 (2017)

  25. Haar, A.: Zur theorie der orthogonalen funktionensysteme. Math. Ann. 69(3), 331–371 (1910)

    Article  MathSciNet  MATH  Google Scholar 

  26. Daugman, J.G.: Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. JOSA A 2(7), 1160–1169 (1985)

    Article  Google Scholar 

  27. Deepu, V., Madhvanath, S., Ramakrishnan, A.G.: Principal component analysis for online handwritten character recognition. In: International Conference on Pattern Recognition, vol. 2, pp. 327–330. IEEE (2004)

  28. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)

  29. Lécun, Y., Bottou, L., Bengio, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (2001)

    Article  Google Scholar 

  30. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(2), 2012 (2013)

    Google Scholar 

  31. Simonyan, K., Zisserman, A.,: Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556 (2014)

  32. Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions. In: Computer Vision and Pattern Recognition, pp. 1–9. IEEE (2015)

  33. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778 (2016)

  34. Deng, J., Dong, W., Socher, R., et al.: ImageNet: a large-scale hierarchical image database. IEEE Conference on IEEE Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 248–255 (2009)

  35. Xu, K., Ba, J., Kiros, R.,: Show, attend and tell: neural image caption generation with visual attention[C]. In: International conference on machine learning, pp. 2048-2057 (2015)

  36. Zhang, H., Goodfellow, I., Metaxas, D., et al.: Self-Attention Generative Adversarial Networks (2018). arXiv preprint arXiv:1805.08318

  37. Oyallon, E., Mallat, S.: Deep roto-translation scattering for object classification. In: Computer Vision and Pattern Recognition. IEEE, pp. 2865–2873 (2015)

  38. Oyallon, E., Belilovsky, E., Zagoruyko, S.: Scaling the scattering transform: deep hybrid networks[C]. In: Proceedings of the IEEE international conference on computer vision, pp. 5618–5627 (2017)

  39. Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), pp. 807–814 (2010)

  40. Ioffe, S., Szegedy, C.: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (2015). arXiv preprint arXiv:1502.03167

  41. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533 (1986)

    Article  MATH  Google Scholar 

  42. Abadi, M., Barham, P., Chen, J., et al.: Tensorflow: a system for large-scale machine learning. In: OSDI, vol. 16, pp. 265–283 (2016)

  43. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  44. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. J. Mach. Learn. Res. 9, 249–256 (2010)

    Google Scholar 

  45. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980 (2014)

  46. Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(2605), 2579–2605 (2008)

    MATH  Google Scholar 

Download references

Acknowledgement

This work is support by National Key Research and Development Plan (No. 2017YFB1402103), Xi’an Science and Technology Bureau Project (201805037YD15CG21(6)). The authors are grateful for the anonymous reviewers for their significant comments.

Author information

Authors and Affiliations

  1. Institute of Computer Science and Engineering, Xi’an University of Technology, Xi’an, China

    Jiulong Zhang & Mingtao Guo

  2. Shaanxi Key Laboratory for Network Computing and Security Technology, Xi’an, China

    Jiulong Zhang

  3. University of North Carolina at Charlotte, Charlotte, USA

    Jianping Fan

Authors
  1. Jiulong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingtao Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianping Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiulong Zhang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Guo, M. & Fan, J. A novel CNN structure for fine-grained classification of Chinese calligraphy styles. IJDAR 22, 177–188 (2019). https://doi.org/10.1007/s10032-019-00324-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-019-00324-1

Keywords

Search

Navigation