Skip to main content
SpringerLink
Log in

Letter-Level Online Writer Identification

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Writer identification (writer-id), an important field in biometrics, aims to identify a writer by their handwriting. Identification in existing writer-id studies requires a complete document or text, limiting the scalability and flexibility of writer-id in realistic applications. To make the application of writer-id more practical (e.g., on mobile devices), we focus on a novel problem, letter-level online writer-id, which requires only a few trajectories of written letters as identification cues. Unlike text-\(\backslash \) document-based writer-id which has rich context for identification, there are much fewer clues to recognize an author from only a few single letters. A main challenge is that a person often writes a letter in different styles from time to time. We refer to this problem as the variance of online writing styles (Var-O-Styles). We address the Var-O-Styles in a capture-normalize-aggregate fashion: Firstly, we extract different features of a letter trajectory by a carefully designed multi-branch encoder, in an attempt to capture different online writing styles. Then we convert all these style features to a reference style feature domain by a novel normalization layer. Finally, we aggregate the normalized features by a hierarchical attention pooling (HAP), which fuses all the input letters with multiple writing styles into a compact feature vector. In addition, we also contribute a large-scale LEtter-level online wRiter IDentification dataset (LERID) for evaluation. Extensive comparative experiments demonstrate the effectiveness of the proposed framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. For convenience of description and simplicity of denotation, we assume that the writer writes the specific letters ‘a’, ‘b’, ..., ‘g’.

  2. In our setting, \(T=64\) and \(d=512\). If we directly flatten \({\mathbf {e}}^{*}_{time}\), the dimension is over 30k.

References

  • Arandjelović, R., & Zisserman, A. (2012) Three things everyone should know to improve object retrieval. In Computer vision and pattern recognition (pp. 2911–2918). IEEE.

  • Bertolini, D., Oliveira, L. S., Justino, E., & Sabourin, R. (2013). Texture-based descriptors for writer identification and verification. Expert Systems with Applications, 40, 2069–2080.

    Article  Google Scholar 

  • Bulacu, M., & Schomaker, L. (2007). Text-independent writer identification and verification using textural and allographic features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 701–717.

    Article  Google Scholar 

  • Carlucci, F.M., Porzi, L., Caputo, B., Ricci, E., & Bulò, S.R. (2017) Just dial: Domain alignment layers for unsupervised domain adaptation. In International conference on image analysis and processing (pp. 357–369). Springer.

  • Chaabouni, A., Boubaker, H., Kherallah, M., Alimi, A.M., El Abed, H. (2011) Multi-fractal modeling for on-line text-independent writer identification. In International conference on document analysis and recognition (pp. 623–627). IEEE.

  • Chen, K. T. (1958). Integration of paths: A faithful representation of paths by noncommutative formal power series. IEEE Transactions of the American Mathematical Society, 89, 395–407.

    MATH  Google Scholar 

  • Chen, Z., Yu, H.X., Wu, A., & Zheng, W.S. (2018) Letter-level writer identification. In Automatic face & gesture recognition pp. 381–388. IEEE.

  • Christlein, V., Maier, A. (2018) Encoding CNN activations for writer recognition. In International association for pattern recognition (pp. 169–174). IEEE.

  • Dhingra, B., Liu, H., Yang, Z., Cohen, W.W., Salakhutdinov, R. (2017) Gated-attention readers for text comprehension. In Meeting of the association for computational linguistics (pp. 1832–1846). ACL.

  • Dwivedi, I., Gupta, S., Venugopal, V., & Sundaram, S. (2016) Online writer identification using sparse coding and histogram based descriptors. In International conference on frontiers in handwriting recognition (pp. 572–577). IEEE.

  • El Abed, H., Märgner, V., Kherallah, M., & Alimi, A.M. (2009) Icdar 2009 online arabic handwriting recognition competition. In International conference on document analysis and recognition (pp. 1388–1392). IEEE.

  • Feng, M., Xiang, B., Glass, M.R., Wang, L., & Zhou, B. (2015) Applying deep learning to answer selection: a study and an open task. In Automatic speech recognition and understanding (pp. 813–820). IEEE.

  • Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., & Lu, H. (2019) Dual attention network for scene segmentation. In Computer vision and pattern recognition (pp. 3146–3154). IEEE.

  • Gargouri, M., Kanoun, S., & Ogier, J.M. (2013) Text-independent writer identification on online arabic handwriting. In International conference on document analysis and recognition (pp. 428–432). IEEE.

  • Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.

    Article  Google Scholar 

  • IAM On-Line Handwriting Database. http://www.fki.inf.unibe.ch/databases.

  • Ioffe, S., & Szegedy, C. (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning (pp. 448–456). IEEE.

  • Khan, F. A., Khelifi, F., Tahir, M. A., & Bouridane, A. (2018). Dissimilarity gaussian mixture models for efficient offline handwritten text-independent identification using sift and rootsift descriptors. IEEE Transactions on Information Forensics and Security, 14, 289–303.

    Article  Google Scholar 

  • Lai, S., & Jin, L. (2019) Offline writer identification based on the path signature feature. In International conference on document analysis and recognition (pp. 1137–1142). IEEE.

  • Lai, S., Jin, L., Lin, L., Zhu, Y., & Mao, H. (2020) Synsig2vec: Learning representations from synthetic dynamic signatures for real-world verification. In Association for the advancement of artificial intelligence (pp. 735–742). AAAI.

  • Li, B., Sun, Z., & Tan, T. (2007) Online text-independent writer identification based on stroke’s probability distribution function. In International conference on biometrics (pp. 201–210). Springer.

  • Li, B., & Tan, T. (2009) Online text-independent writer identification based on temporal sequence and shape codes. In International conference on document analysis and recognition (pp. 931–935). IEEE.

  • Lin, Z., Feng, M., Santos, C. N. D., Yu, M., Xiang, B., Zhou, B., & Bengio, Y. (2017) A structured self-attentive sentence embedding. In International conference on learning representations.

  • Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., & Song, L. (2017) Sphereface: Deep hypersphere embedding for face recognition. In Computer vision and pattern recognition (pp. 212–220). IEEE.

  • Maaten, L.v.d., Hinton, G., (2008). Visualizing data using t-SNE. Journal of Machine Learning Research, 9, 2579–2605.

  • Moon, H., & Phillips, P. J. (2001). Computational and performance aspects of pca-based face-recognition algorithms. Perception, 30, 303–321.

    Article  Google Scholar 

  • Nam, H., Ha, J.W., & Kim, J. (2017) Dual attention networks for multimodal reasoning and matching. In Computer vision and pattern recognition (pp. 299–307). IEEE.

  • Namboodiri, A., & Gupta, S. (2006) Text independent writer identification from online handwriting. In International conference on frontiers in handwriting recognition (pp. 566–571).

  • Nasuno, R., Arai, S. (2017) Writer identification for offline japanese handwritten character using convolutional neural network. In International conference on intelligent systems and image processing (pp. 94–97). Springer.

  • Nguyen, H. T., Nguyen, C. T., Ino, T., Indurkhya, B., & Nakagawa, M. (2019). Text-independent writer identification using convolutional neural network. Pattern Recognition, 121, 104–112.

    Article  Google Scholar 

  • Ramaiah, C., Shivram, A., & Govindaraju, V. (2013) A bayesian framework for modeling accents in handwriting. In International conference on document analysis and recognition (pp. 917–921). IEEE.

  • Sae-Bae, N., & Memon, N. (2014). Online signature verification on mobile devices. IEEE Transactions on Information Forensics and Security, 9, 933–947.

    Article  Google Scholar 

  • Schlapbach, A., & Bunke, H. (2007) Fusing asynchronous feature streams for on-line writer identification. In International conference on document analysis and recognition (Vol. 1, pp. 103–107). IEEE.

  • Schlapbach, A., Liwicki, M., & Bunke, H. (2008). A writer identification system for on-line whiteboard data. Pattern Recognition, 41(7), 2381–2397.

    Article  MATH  Google Scholar 

  • Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. Signal Processing, 45, 2673–2681.

    Google Scholar 

  • Shivram, A., Ramaiah, C., & Govindaraju, V. (2013). A hierarchical bayesian approach to online writer identification. Iet Biometrics, 2(4), 191–198.

    Article  Google Scholar 

  • Si, J., Zhang, H., Li, C.G., Kuen, J., Kong, X., Kot, A.C., & Wang, G. (2018) Dual attention matching network for context-aware feature sequence based person re-identification. In Computer vision and pattern recognition (pp. 5363–5372). IEEE.

  • Simonyan, K., & Zisserman, A. (2015) Very deep convolutional networks for large-scale image recognition. In International conference on learning representations.

  • Singh, G., & Sundaram, S. (2015) A subtractive clustering scheme for text-independent online writer identification. In International conference on document analysis and recognition (pp. 311–315). IEEE.

  • Song, S., Lan, C., Xing, J., Zeng, W., & Liu, J. (2017) An end-to-end spatio-temporal attention model for human action recognition from skeleton data.

  • Tan, G. X., Viard-Gaudin, C., & Kot, A. C. (2010). Individuality of alphabet knowledge in online writer identification. International Journal on Document Analysis and Recognition, 13(2), 147–157.

    Article  Google Scholar 

  • Tang, Y., & Wu, X. (2016) Text-independent writer identification via CNN features and joint bayesian. In International conference on frontiers in handwriting recognition (pp. 566–571). IEEE.

  • Tieleman, T., & Hinton, G. (2012). Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. coursera: Neural networks for machine learning, 4(2), 26–31.

    Google Scholar 

  • Tsai, M.Y., & Lan, L.S. (2005) Online writer identification using the point distribution model. In International conference on systems, man, and cybernetics society (pp. 1264–1268). IEEE.

  • Venugopal, V., & Sundaram, S. (2017). An online writer identification system using regression-based feature normalization and codebook descriptors. Expert Systems with Applications, 72, 196–206.

    Article  Google Scholar 

  • Venugopal, V., & Sundaram, S. (2018). An improved online writer identification framework using codebook descriptors. Pattern Recognition, 78, 318–330.

    Article  Google Scholar 

  • Venugopal, V., & Sundaram, S. (2018) Modified sparse representation classification framework for online writer identification. IEEE Transactions on Systems, Man, and Cybernetics: Systems, pp. 1–12.

  • Venugopal, V., & Sundaram, S. (2018). Online writer identification with sparse coding-based descriptors. IEEE Transactions on Information Forensics and Security, 13(10), 2538–2552.

    Article  Google Scholar 

  • Vorugunti, C.S., Mukherjee, P., Guru, D.S., & Pulabaigari, V. (2019) Online signature verification based on writer specific feature selection and fuzzy similarity measure. In Computer vision and pattern recognition (pp. 88–95). IEEE.

  • Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., Wang, X., & Tang, X. (2017) Residual attention network for image classification. In Computer vision and pattern recognition (pp. 3156–3164). IEEE.

  • Wang, F., Xiang, X., Cheng, J., Yuille, A.L. (2017) Normface: L2 hypersphere embedding for face verification. In International conference on multimedia (pp. 1041–1049). ACM.

  • Xing, L., Qiao, Y. (2016) Deepwriter: A multi-stream deep CNN for text-independent writer identification. In International conference on frontiers in handwriting recognition (pp. 584–589). IEEE.

  • Xing, Z. J., Yin, F., Wu, Y. C., & Liu, C. L. (2018). Offline signature verification using convolution siamese network. International conference on graphic and image processing (pp. 415–423). SPIE: International Society for Optics and Photonics.

    Google Scholar 

  • Yang, W., Jin, L., & Liu, M. (2016). DeepWriterID: An end-to-end online text-independent writer identification system. Intelligent Systems, 31, 45–53.

    Article  Google Scholar 

  • Zhang, X. Y., Xie, G. S., Liu, C. L., & Bengio, Y. (2017). End-to-end online writer identification with recurrent neural network. IEEE Transactions on Human-Machine Systems, 47, 285–292.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported partially by the National Key Research and Development Program of China (2018YFB1004903), NSFC(U1911401,U1811461), Guangdong Province Science and Technology Innovation Leading Talents (2016TX03X157), Guangdong NSF Project (No. 2018B030312002), Guangzhou Research Project (201902010037), and Research Projects of Zhejiang Lab (No. 2019KD0AB03), and the Key-Area Research and Development Program of Guangzhou (202007030004). The corresponding author and principal investigator for this paper is Wei-Shi Zheng.

Author information

Authors and Affiliations

  1. School of Data and Computer Science, Sun Yat-set University, Guangzhou, China

    Zelin Chen, Hong-Xing Yu, Ancong Wu & Wei-Shi Zheng

Authors
  1. Zelin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hong-Xing Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ancong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei-Shi Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei-Shi Zheng.

Additional information

Communicated by Boxin Shi.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Z., Yu, HX., Wu, A. et al. Letter-Level Online Writer Identification. Int J Comput Vis 129, 1394–1409 (2021). https://doi.org/10.1007/s11263-020-01414-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-020-01414-y

Keywords

Search

Navigation