Skip to main content
SpringerLink
Log in

Fusion of LLE and stochastic LEM for Persian handwritten digits recognition

  • Original Paper
  • Published:
International Journal on Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

In this paper, a new local manifold learning (ML) method is proposed. Our proposed method, which is named FSLL, is based on the fusion of locally linear embedding (LLE) and a new Stochastic Laplacian Eigenmaps (SLEM). SLEM is the same as a common LEM technique, but the coefficients between each data point and its neighbors are calculated by a stochastic process. The coefficients of SLEM make a probability mass function scheme, and their entropy is set to a certain value. The entropy value is an estimation of the locality around each data point. Two criteria will be presented based on the mutual neighborhood conception to determine the entropy value. In LLE, each data point is linearly reconstructed based on its neighbors and then the embedded data manifold is extracted by preserving these linear reconstruction coefficients. LLE and SLEM extract and learn the embedded data manifold by two different kinds of local structure information. In FSLL, two local ML methods, SLEM and LLE, are fused by rewriting their cost functions without the need for any projection space. Fusion of these two techniques provides more structural information at high-dimensional space that can be applied on extracting the embedded low-dimensional data. Also, in this study, a feature vector will be presented by combining a HMAX feature vector and a PCA-based feature vector. Evaluations of the proposed method are done on Persian handwritten digit IFHCDB and IPHD databases in image and feature spaces. The results demonstrate the performance of FSLL and SLEM. The recognition rates are improved about 4% in most dimensionalities. Also, a method of out-of-sample test data extension is proposed corresponding to the proposed methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Deng, L., Yu, D.: Deep learning: methods and applications. Found. Trends Signal Process. 7(3–4), 197–387 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  2. Belkin, M., Niyogi, P.: Laplacian Eigenmaps and spectral techniques for embedding and clustering. Adv Neural Inf Process Syst 14, 585–591 (2001)

    Google Scholar 

  3. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by Locally Linear Embedding. Science 290(5500), 2323–2326 (2000)

    Article  Google Scholar 

  4. He, X., Niyogi, P.: Locality preserving projections. In: NIPS, vol. 16, No. 2003 (2003)

  5. Donoho, D.L., Grimes, C.: Hessian eigenmaps: Locally Linear Embedding techniques for high-dimensional data. Proce. Natl. Acad. Sci. 100(10), 5591–5596 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  6. Zhang, Z.Y., Zha, H.Y.: Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. J. Shanghai Univ. 8(4), 406–424 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  7. Hinton, G., Roweis, S.: Stochastic neighbor embedding. In: NIPS, vol. 15, pp. 833–840 (2002)

  8. Jolliffe, I.: Principal Component Analysis. Wiley, London (2002)

    MATH  Google Scholar 

  9. Weinberger, K.Q., Sha, F., Saul, L:K.: Learning a kernel matrix for nonlinear dimensionality reduction. In: Proceedings of the Twenty-First International Conference on Machine Learning 2004 Jul 4, p. 106 (2004)

  10. Cox, T.F., Cox, M.A.: Multidimensional Scaling. CRC Press, Boca Raton (2000)

    MATH  Google Scholar 

  11. Kohonen, T.: The self-organizing map. Neurocomputing 21(1), 1–6 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  12. Tenenbaum, J.B., Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)

    Article  Google Scholar 

  13. Ma, L., Crawford, M.M., Yang, X., Guo, Y.: Local-manifold-learning-based graph construction for semisupervised hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 53(5), 2832–2844 (2015)

    Article  Google Scholar 

  14. Orsenigo, C., Vercellis, C.: A comparative study of nonlinear manifold learning methods for cancer microarray data classification. Expert Syst. Appl. 40(6), 2189–2197 (2013)

    Article  Google Scholar 

  15. Talmon, R., Mallat, S., Zaveri, H., Coifman, R.R.: Manifold learning for latent variable inference in dynamical systems. IEEE Trans. Signal Process. 63(15), 3843–3856 (2015)

    Article  MathSciNet  Google Scholar 

  16. Chahooki, M.A.Z., Charkari, N.M.: Shape classification by manifold learning in multiple observation spaces. Inform. Sci. 262, 46–61 (2014)

    Article  MATH  Google Scholar 

  17. . Imani, M., Ghassemian, H.: A manifold learning based feature extraction method with improved discriminative ability. In: 2015 9th Iranian Conference on Machine Vision and Image Processing (MVIP), pp. 29–32 (2015)

  18. Naz, S., Hayat, K., Razzak, M.I., Anwar, M.W., Madani, S.A., Khan, S.U.: The optical character recognition of Urdu-like cursive scripts. Pattern Recognit. 47(3), 1229–1248 (2014)

    Article  Google Scholar 

  19. Askari, M., Asadi, M., Asilian Bidgoli, A., Ebrahimpour, H.: Isolated Persian/Arabic handwriting characters: derivative projection profile features, implemented on GPUs. J. AI Data Min. 4(1), 9–17 (2016)

    Google Scholar 

  20. Ghods, V., Sohrabi, M.K.: Online Farsi handwritten character recognition using hidden Markov model. JCP 11(2), 169–175 (2016)

    Article  Google Scholar 

  21. . Razavi, S.M., Kabir, E.: A data set for online Farsi handwriting. In: Proceedings of the 6th National Conference on Intelligent Systems (in Farsi), pp. 218–225 (2004)

  22. Hussain, R., Raza, A., Siddiqi, I., Khurshid, K., Djeddi, C.: A comprehensive survey of handwritten document benchmarks: structure, usage and evaluation. EURASIP J. Image Video Process. 2015(1), 46 (2015)

    Article  Google Scholar 

  23. Cheriet, M., Moghaddam, R.F., Arabnejad, E., Zhong, G.: Manifold learning for the shape-based recognition of historical Arabic documents. Handbook of Statistics: Machine Learning—Theory and Applications 31, 471 (2013)

    Article  MathSciNet  Google Scholar 

  24. Tao, D., Liang, L., Jin, L., Gao, Y.: Similar handwritten Chinese character recognition using discriminative locality alignment manifold learning. In: 2011 International Conference on Document Analysis and Recognition (ICDAR), pp. 1012–1016 (2011)

  25. Zhong, G., Chherawala, Y., Cheriet, M.: An empirical evaluation of supervised dimensionality reduction for recognition. In: 2013 12th International Conference on Document Analysis and Recognition (ICDAR), pp. 1315–1319 (2013)

  26. Xing, X., Wang, K., Lv, Z., Zhou, Y., Du, S.: Fusion of local manifold learning methods. IEEE Signal Process. Lett. 22(4), 395–399 (2015)

    Article  Google Scholar 

  27. Gan, Q., Shen, F., Zhao, J.: Improved Manifold Learning with competitive Hebbian rule. In: 2015 International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2015)

  28. Wang, J., Zhang, Z., Zha, H.: Adaptive manifold learning. In: NIPS, vol. 2004 (2004)

  29. Wei, J., Peng, H., Lin, Y.S., Huang, Z.M., Wang, J.B.: Adaptive neighborhood selection for manifold learning. In: 2008 International Conference on Machine Learning and Cybernetics, vol. 1, pp. 380–384 (2008)

  30. Li, B., Zhang, Y.: Supervised Locally Linear Embedding projection (SLLEP) for machinery fault diagnosis. Mech. Syst. Signal Process. 25(8), 3125–3134 (2011)

    Article  Google Scholar 

  31. Zhao, X., Zhang, S.: Facial expression recognition using local binary patterns and discriminant kernel Locally Linear Embedding. EURASIP J. Adv. Signal Process. 2012, 1–9 (2012)

    Article  Google Scholar 

  32. Maaten, L.V.D., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)

    MATH  Google Scholar 

  33. Hajizadeh, R., Aghagolzadeh, A., Ezoji, M.: Manifold based Persian digit recognition using the modified Locally Linear Embedding and linear discriminative analysis. In: 2015 2nd International Conference on Knowledge-Based Engineering and Innovation (KBEI), pp. 614–618 (2015)

  34. Bengio, Y., Paiement, J.F., Vincent, P., Delalleau, O., Le Roux, N., Ouimet, M.: Out-of-Sample Extensions for Lle, Isomap, Mds, Eigenmaps, and Spectral clustering. Advances in Neural Information Processing Systems, vol. 16. MIT Press, Cambridge (2003)

    Google Scholar 

  35. Ziaratban, M., Faez, K., Faradji, F.: Language-based feature extraction using template-matching in Farsi/Arabic handwritten numeral recognition. In: Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), vol. 1, pp. 297–301 (2007)

  36. Mozaffari, S., Faez, K., Faradji, F., Ziaratban, M., Golzan, S.M.: A comprehensive isolated Farsi/Arabic character database for handwritten OCR research. In: Tenth International Workshop on Frontiers in Handwriting Recognition, Suvisoft (2006)

  37. Sajedi, H., Bahador, M.: Persian handwritten number recognition using adapted framing feature and support vector machines. Int. J. Comput. Intell. Appl. 15(01), 1650004 (2016)

    Article  Google Scholar 

  38. Poggio, T., Knoblich, U., Mutch, J.: CNS: a GPU-based framework for simulating cortically-organized networks. Computer Science and Artificial Intelligence Laboratory, Technical Report (2010)

Download references

Author information

Authors and Affiliations

  1. Faculty of Electrical and Computer Engineering, Babol Noshirvani University of Technology, Babol, Iran

    Rassoul Hajizadeh, A. Aghagolzadeh & M. Ezoji

Authors
  1. Rassoul Hajizadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Aghagolzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Ezoji
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. Aghagolzadeh.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hajizadeh, R., Aghagolzadeh, A. & Ezoji, M. Fusion of LLE and stochastic LEM for Persian handwritten digits recognition. IJDAR 21, 109–122 (2018). https://doi.org/10.1007/s10032-018-0303-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-018-0303-4

Keywords

Search

Navigation