Abstract
In this paper, we focus on building an optimization scheme over the Stiefel manifold that maintains each iterate feasible. We focus on conjugate gradient methods and compare our scheme to the Riemannian optimization approach. We parametrize the Stiefel manifold using the polar decomposition to build an optimization problem over a vector space, instead of a Riemannian manifold. The result is a conjugate gradient method that averts the use of a vector transport, needed in the Riemannian conjugate gradient method. The performance of our method is tested on a variety of numerical experiments and compared with those of three Riemannian optimization methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abrudan T, Eriksson J, Koivunen V (2009) Conjugate gradient algorithm for optimization under unitary matrix constraint. Sig Process 89(9):1704–1714
Abrudan TE, Eriksson J, Koivunen V (2008) Steepest descent algorithms for optimization under unitary matrix constraint. IEEE Trans Signal Process 56(3):1134–1147
Absil PA, Baker CG, Gallivan KA (2007) Trust-region methods on riemannian manifolds. Found Comput Math 7(3):303–330
Absil PA, Mahony R, Sepulchre R (2009) Optimization algorithms on matrix manifolds. Princeton University Press, Princeton
Baker CG, Absil PA, Gallivan KA (2008) An implicit trust-region method on riemannian manifolds. IMA J Numer Anal 28(4):665–689
Balogh J, Csendes T, Rapcsák T (2004) Some global optimization problems on stiefel manifolds. J Global Optim 30(1):91–101
Dai Y (2002) Nonmonotone conjugate gradient algorithm for unconstrained optimization. J Syst Sci Complexity 15(2):139–145
Dalmau O, Leon HFO (2017) Projected nonmonotone search methods for optimization with orthogonality constraints. Comput Appl Math pp 1–27
Eldén L, Park H (1999) A procrustes problem on the stiefel manifold. Numer Math 82(4):599–619
Fackler PL (2005) Notes on matrix calculus. North Carolina State University, North Carolina
Francisco J, Martini T (2014) Spectral projected gradient method for the procrustes problem. TEMA (São Carlos) 15(1):83–96
Golub GH, van Loan CF (2013) Matrix Computations, 4th edn. JHU Press. URL http://www.cs.cornell.edu/cv/GVL4/golubandvanloan.htm
Gower JC, Dijksterhuis GB (2004) Procrustes problems, vol 30. Oxford University Press on Demand, Oxford
Grubišić I, Pietersz R (2007) Efficient rank reduction of correlation matrices. Linear Algebra Appl 422(2–3):629–653
Huang W, Absil PA, Gallivan KA (2017) Intrinsic representation of tangent vectors and vector transports on matrix manifolds. Numer Math 136(2):523–543
Huang W, Gallivan KA, Absil PA (2015) A broyden class of quasi-newton methods for riemannian optimization. SIAM J Optim 25(3):1660–1685
Joho M, Mathis H (2002) Joint diagonalization of correlation matrices by using gradient methods with application to blind signal separation. In: Sensor array and multichannel signal processing workshop proceedings, IEEE, pp 273–277
Li Q, Qi H (2011) A sequential semismooth newton method for the nearest low-rank correlation matrix problem. SIAM J Optim 21(4):1641–1666
Liu X, Wen Z, Wang X, Ulbrich M, Yuan Y (2015) On the analysis of the discretized kohn-sham density functional theory. SIAM J Numer Anal 53(4):1758–1785
Manton JH (2002) Optimization algorithms exploiting unitary constraints. IEEE Trans Signal Process 50(3):635–650
Nesterov Y (1983) A method for solving the convex programming problem with convergence rate O\((1/k^2)\). Sov Math Doklady 27:372–376
Nishimori Y, Akaho S (2005) Learning algorithms utilizing quasi-geodesic flows on the stiefel manifold. Neurocomputing 67:106–135
Nocedal J, Wright SJ (2006) Numerical optimization. Springer series in operations research and financial engineering, 2nd edn. Springer, New York
Pietersz R, Groenen PJ (2004) Rank reduction of correlation matrices by majorization. Quant Financ 4(6), 649–662
Ring W, Wirth B (2012) Optimization methods on riemannian manifolds and their application to shape space. SIAM J Optim 22(2):596–627
Saad Y (1992) Numerical methods for large eigenvalue problems. Manchester University Press, Manchester
Sato H, Iwai T (2015) A new, globally convergent riemannian conjugate gradient method. Optimization 64(4):1011–1031
Savas B, Lim LH (2010) Quasi-newton methods on grassmannians and multilinear approximations of tensors. SIAM J Sci Comput 32(6):3352–3393
Schönemann PH (1966) A generalized solution of the orthogonal procrustes problem. Psychometrika 31(1):1–10
Theis FJ, Cason TP, Absil PA (2009) Soft dimension reduction for ica by joint diagonalization on the stiefel manifold. In: International conference on independent component analysis and signal separation, Springer, New York, pp 354–361
Wen Z, Yang C, Liu X, Zhang Y (2016) Trace-penalty minimization for large-scale eigenspace computation. J Sci Comput 66(3):1175–1203
Wen Z, Yin W (2013) A feasible method for optimization with orthogonality constraints. Math Program 142(1–2):397–434
Yang C, Meza JC, Lee B, Wang LW (2009) Kssolv–a matlab toolbox for solving the kohn-sham equations. ACM Trans Math Softw (TOMS) 36(2):10
Yang C, Meza JC, Wang LW (2007) A trust region direct constrained minimization algorithm for the kohn-sham equation. SIAM J Sci Comput 29(5):1854–1875
Zhu X (2015) A feasible filter method for the nearest low-rank correlation matrix problem. Numer Algorithm 69(4):763–784
Zhu X (2017) A riemannian conjugate gradient method for optimization on the stiefel manifold. Comput Optim Appl 67(1):73–110
Acknowledgements
This work was supported in part by CONACYT (Mexico), Grant 258033.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Joerg Fliege.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Figueroa, E.F., Dalmau, O. Transportless conjugate gradient for optimization on Stiefel manifold. Comp. Appl. Math. 39, 151 (2020). https://doi.org/10.1007/s40314-020-01184-w
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s40314-020-01184-w