Skip to main content
SpringerLink
Log in

Multiplicative Algorithms for Symmetric Nonnegative Tensor Factorizations and Its Applications

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

Nonnegative tensor factorization (NTF) and nonnegative Tucker decomposition (NTD) have been widely applied in high-dimensional nonnegative tensor data analysis. This paper focuses on symmetric NTF and symmetric NTD, which are the special cases of NTF and NTD, respectively. By minimizing the Euclidean distance and the generalized KL divergence, the multiplicative updating rules are proposed and the convergence under mild conditions is proved. We also show that if the solution converges based on the multiplicative updating rules, then the limit satisfies the Karush–Kuhn–Tucker optimality conditions. We illustrate the efficiency of these multiplicative updating rules via several numerical examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. The Yale Face Database is at http://vision.ucsd.edu/content/extended-yale-face-database-b-b.

References

  1. Agarwal, S., Branson, K., Belongie,S.: Higher order learning with graphs. In: International Conference on Mechine Learning, pp. 1150–1157. (2006)

  2. Bader, B.W., Kolda, T.G.: Matlab tensor toolbox version 2.6. http://www.sandia.gov/~tgkolda/TensorToolbox/. Accessed 6 Feb 2015

  3. Carroll, J.D., Chang, J.-J.: Analysis of individual differences in multidimensional scaling via an \(n\)-way generalization of “Eckart–Young” decomposition. Psychometrika 35, 283–319 (1970)

    Article  Google Scholar 

  4. Che, M., Bu, C., Qi, L., Wei, Y.: Nonnegative tensors revisited: plane stochastic tensors. Linear Multilinear A. 67, 1364–1391 (2019)

    Article  MathSciNet  Google Scholar 

  5. Chen, G., Lerman, G.: Spectral curvature clustering (SCC). Int. J. Comput. Vis. 81, 317–330 (2009)

    Article  Google Scholar 

  6. Chen, Y., Rege, M., Dong, M., Hua, J.: Nonnegative matrix factorization for semi-supervised data clustering. Knowl. Inf. Syst. 17, 355–379 (2008)

    Article  Google Scholar 

  7. Chi, E.C., Kolda, T.G.: On tensors, sparsity, and nonnegative factorizations. SIAM J. Matrix Anal. Appl. 33, 1272–1299 (2012)

    Article  MathSciNet  Google Scholar 

  8. Cichocki, A., Zdunek, R., Amari, S.: Csiszárs divergences for non-negative matrix factorization: family of new algorithms. Lect. Notes Comput. Sci. 3889, 32–39 (2006)

    Article  Google Scholar 

  9. Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.-I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley, Hoboken (2009)

    Book  Google Scholar 

  10. De Lathauwer, L., De Moor, B., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21, 1253–1278 (2000)

    Article  MathSciNet  Google Scholar 

  11. Ding, C., He, X., Simon, H.D.: On the equivalence of nonnegative matrix factorization and spectral clustering. In: Proceedings of SIAM International Conference on Data Mining, pp. 606–610. (2005)

  12. Fan, J., Zhou, A.: A semidefinite algorithm for completely positive tensor decomposition. Comput. Optim. Appl. 66, 267–283 (2017)

    Article  MathSciNet  Google Scholar 

  13. Fevotte, C., Bertin, N., Durrieu, J.: Nonnegative matrix factorization with the Itakura–Saito divergence: with application to music analysis. Neural Comput. 21, 793–830 (2009)

    Article  Google Scholar 

  14. Georghiades, A.S., Belhumeur, P.N., Kriegman, D.: From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell. 23, 643–660 (2001)

    Article  Google Scholar 

  15. Golub, G.H., Van Loan, C.F.: Matrix Computations, Johns Hopkins Studies in the Mathematical Sciences, 4th edn. Johns Hopkins University Press, Baltimore (2013)

    Google Scholar 

  16. Govindu, V.M.: A tensor decomposition for geometric grouping and segmentation. In: IEEE CAMSAP, 2005: Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1150–1157. IEEE (2005)

  17. Hansen, S., Plantenga, T.D., Kolda, T.G.: Newton-based optimization for Kullback–Leibler non-negative tensor factorizations. Optim. Methods Softw. 30, 1002–1029 (2015)

    Article  MathSciNet  Google Scholar 

  18. He, Z., Cichocki, A., Xie, S., Choi, K.: Detecting the number of clusters in n-way probabilistic clustering. IEEE Trans. Pattern Anal. Mach. Intell. 32, 2006–2021 (2010)

    Article  Google Scholar 

  19. He, Z., Xie, S., Zdunek, R., Zhou, G., Cichocki, A.: Symmetric nonnegative matrix factorization: algorithms and applications to probabilistic clustering. IEEE Trans. Neural Netw. 22, 2117–2131 (2011)

    Article  Google Scholar 

  20. Ho, J., Yang, M., Lim, J., Lee, K., Kriegman, D., Clustering appearances of objects under varing illustration conditions. In: Proceedings of the IEEE CS Conference on Computer Vision and Pattern Recognitions, vol. 1, pp. 11–18. (2003)

  21. Kim, J., He, Y., Park, H.K.: Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework. J. Glob. Optim. 58, 285–319 (2014)

    Article  MathSciNet  Google Scholar 

  22. Kim, Y.D., Choi, S.: Nonnegative tucker decomposition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1–8. (2007)

  23. Kolda, T.G.: Numerical optimization for symmetric tensor decomposition. Math. Program. 151, 225–248 (2015)

    Article  MathSciNet  Google Scholar 

  24. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51, 455–500 (2009)

    Article  MathSciNet  Google Scholar 

  25. Lee, D., Seung, H.S.: Algorithms for Non-negative Matrix Factorization. In: Neural Information Processing Systems, pp. 556–562. (2000)

  26. Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)

    Article  Google Scholar 

  27. Liavas, A.P., Kostoulas, G., Lourakis, G., Huang, K., Sidiropoulos, N.D.: Nesterov-based alternating optimization for nonnegative tensor factorization: algorithm and parallel implementation. IEEE Trans. Signal Process. 66, 944–953 (2018)

    Article  MathSciNet  Google Scholar 

  28. Lim, L.: Singular values and eigenvalues of tensors: a variational approach. In: IEEE CAMSAP, 2005: First International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, pp. 129–132. IEEE (2005)

  29. Lim, L.-H., Comon, P.: Nonnegative approximations of nonnegative tensors. J. Chemom. 23, 432–441 (2009)

    Article  Google Scholar 

  30. Long, B., Zhang, Z., Wu, X., Yu, P.S.: Relational clustering by symmetric convex coding. In: Machine Learning, Proceedings of the Twenty-Fourth International Conference, pp. 569–576. (2007)

  31. Lu, H., Plataniotis, K.N., Venetsanopoulos, A.: Multilinear Subspace Learning: Dimensionality Reduction of Multidimensional Data. CRC Press, Boca Raton (2013)

    Book  Google Scholar 

  32. Luo, Z., Qi, L.: Completely positive tensors: properties, easily checkable subclasses, and tractable relaxations. SIAM J. Matrix Anal. Appl. 37, 1675–1698 (2016)

    Article  MathSciNet  Google Scholar 

  33. Mørup, M., Hansen, L.K., Arnfred, S.M.: Algorithms for sparse nonnegative tucker decompositions. Neural Comput. 20, 2112–2131 (2008)

    Article  Google Scholar 

  34. Nocedal, J., Wright, S.J.: Numerical Optimization, Springer Series in Operations Research. Springer-Verlag, New York (1999)

    Google Scholar 

  35. Qi, L.: Eigenvalues of a real supersymmetric tensor. J. Symb. Comput. 40, 1302–1324 (2005)

    Article  MathSciNet  Google Scholar 

  36. Qi, L., Luo, Z.: Tensor Analysis. Spectral Theory and Special Tensors. Society for Industrial and Applied Mathematics, Philadelphia (2018)

    Google Scholar 

  37. Qi, L., Xu, C., Xu, Y.: Nonnegative tensor factorization, completely positive tensors, and a hierarchical elimination algorithm. SIAM J. Matrix Anal. Appl. 35, 1227–1241 (2014)

    Article  MathSciNet  Google Scholar 

  38. Shashua, A., Zass, R., Hazan, T.: Multi-way clustering using super-symmetric non-negative tensor factorization. Lect. Notes Comput. Sci. 3954, 595–608 (2006)

    Article  Google Scholar 

  39. Song, Y., Qi, L.: Infinite and finite dimensional Hilbert tensors. Linear Algebra Appl. 451, 1–14 (2014)

    Article  MathSciNet  Google Scholar 

  40. Tucker, L.R.: Some mathematical notes on three-mode factor analysis. Psychometrika 31, 279–311 (1966)

    Article  MathSciNet  Google Scholar 

  41. Xu, C., Luo, Z., Qi, L., Chen, Z.: \(\{0,1\}\) completely positive tensors and multi-hypergraphs. Linear Algebra Appl. 510, 110–123 (2016)

    Article  MathSciNet  Google Scholar 

  42. Xu, Y., Yin, W.: A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J. Imaging Sci. 6, 1758–1789 (2013)

    Article  MathSciNet  Google Scholar 

  43. Zass, R., Shashua, A.: A unifying approach to hard and probabilistic clustering. In: International Conference on Computer Vision, pp. 294–301. (2005)

  44. Zhou, G., Cichocki, A., Xie, S.: Fast nonnegative matrix/tensor factorization based on low-rank approximation. IEEE Trans. Signal Process. 60, 2928–2940 (2012)

    Article  MathSciNet  Google Scholar 

  45. Zhou, G., Cichocki, A., Zhao, Q., Xie, S.: Efficient nonnegative tucker decompositions: algorithms and uniqueness. IEEE Trans. Image Process. 24, 4990–5003 (2015)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We would like to thank the Editor-in-Chief, Prof. Chi-Wang Shu and two anonymous reviewers for very helpful comments.

Author information

Authors and Affiliations

  1. School of Economic Mathematics, Southwestern University of Finance and Economics, Chengdu, 611130, People’s Republic of China

    Maolin Che

  2. School of Mathematical Sciences and Key Laboratory of Mathematics for Nonlinear Sciences, Fudan University, Shanghai, 200433, People’s Republic of China

    Yimin Wei

Authors
  1. Maolin Che
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yimin Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yimin Wei.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

M. Che: This author is supported by the National Natural Science Foundation of China under grant 11901471.

Y. Wei: This author is supported by the National Natural Science Foundation of China under Grant 11771099 and Innovation Program of Shanghai Municipal Education Commission.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Che, M., Wei, Y. Multiplicative Algorithms for Symmetric Nonnegative Tensor Factorizations and Its Applications. J Sci Comput 83, 53 (2020). https://doi.org/10.1007/s10915-020-01233-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-020-01233-w

Keywords

Mathematics Subject Classification

Search

Navigation