A geometric approach to subspace updates and orthogonal matrix decompositions under rank-one modifications

Zimmermann, Ralf

doi:10.1090/mcom/3574

A geometric approach to subspace updates and orthogonal matrix decompositions under rank-one modifications
HTML articles powered by AMS MathViewer

by Ralf Zimmermann HTML | PDF

Math. Comp. 90 (2021), 671-688 Request permission

Abstract:

For a matrix $X\in \mathbb {R}^{n\times p}$, we provide an analytic formula that keeps track of an orthonormal basis for the range of $X$ under rank-one modifications. More precisely, we consider rank-one adaptations $X_{new} = X+ab^T$ of a given $X$ with known matrix factorization $X = UW$, where $U\in \mathbb {R}^{n\times p}$ is column-orthogonal and $W\in \mathbb {R}^{p\times p}$ is invertible. Arguably, the most important methods that produce such factorizations are the singular value decomposition (SVD), where $X=UW=U(\Sigma V^T)$, and the QR-decomposition, where $X = UW = QR$. We give a geometric description of rank-one adaptations and derive a closed-form expression for the geodesic line that travels from the subspace $\mathcal {S}= \textrm {{ran}}(X)$ to the subspace $\mathcal {S}_{new} =\textrm {{ran}}(X_{new}) =\textrm {{ran}}(U_{new}W_{new})$. This leads to update formulas for orthogonal matrix decompositions, where both $U_{new}$ and $W_{new}$ are obtained via elementary rank-one matrix updates in $\mathcal {O}(np)$ time for $n\gg p$. Moreover, this allows us to determine the subspace distance and the Riemannian midpoint between the subspaces $\mathcal {S}$ and $\mathcal {S}_{new}$ without additional computational effort.

References

P.-A. Absil, R. Mahony, and R. Sepulchre, Optimization algorithms on matrix manifolds, Princeton University Press, Princeton, NJ, 2008. With a foreword by Paul Van Dooren. MR 2364186, DOI 10.1515/9781400830244
L. Balzano, R. Nowak, and B. Recht, Online identification and tracking of subspaces from highly incomplete information, Communication, control, and computing (allerton), 2010 48th annual allerton conference on, 2010, pp. 704–711.
L. Balzano and S. J. Wright, On GROUSE and incremental SVD, 2013 5th IEEE international workshop on computational advances in multi-sensor adaptive processing (camsap), 2013, pp. 1–4.
Laura Balzano and Stephen J. Wright, Local convergence of an algorithm for subspace identification from partial data, Found. Comput. Math. 15 (2015), no. 5, 1279–1314. MR 3394711, DOI 10.1007/s10208-014-9227-7
Rajendra Bhatia, Positive definite matrices, Princeton Series in Applied Mathematics, Princeton University Press, Princeton, NJ, 2007. MR 2284176
Matthew Brand, Fast low-rank modifications of the thin singular value decomposition, Linear Algebra Appl. 415 (2006), no. 1, 20–30. MR 2214744, DOI 10.1016/j.laa.2005.07.021
James R. Bunch and Christopher P. Nielsen, Updating the singular value decomposition, Numer. Math. 31 (1978/79), no. 2, 111–129. MR 509670, DOI 10.1007/BF01397471
James R. Bunch, Christopher P. Nielsen, and Danny C. Sorensen, Rank-one modification of the symmetric eigenproblem, Numer. Math. 31 (1978/79), no. 1, 31–48. MR 508586, DOI 10.1007/BF01396012
S. Chandrasekaran, B. S. Manjunath, Y. F. Wang, J. Winkeler, and H. Zhang, An eigenspace update algorithm for image analysis, Graphical Models and Image Processing 59 (1997), no. 5.
Moody T. Chu, Robert E. Funderlic, and Gene H. Golub, A rank-one reduction formula and its applications to matrix factorizations, SIAM Rev. 37 (1995), no. 4, 512–530. MR 1368385, DOI 10.1137/1037124
J. W. Daniel, W. B. Gragg, L. Kaufman, and G. W. Stewart, Reorthogonalization and stable algorithms for updating the Gram-Schmidt $QR$ factorization, Math. Comp. 30 (1976), no. 136, 772–795. MR 431641, DOI 10.1090/S0025-5718-1976-0431641-8
M. P. do Carmo, Riemannian Geometry, Mathematics: Theory & Applications, Birkhäuser Boston, 1992.
Alan Edelman, Tomás A. Arias, and Steven T. Smith, The geometry of algorithms with orthogonality constraints, SIAM J. Matrix Anal. Appl. 20 (1999), no. 2, 303–353. MR 1646856, DOI 10.1137/S0895479895290954
A. Galántai, The rank reduction procedure of Egerváry, Central European Journal of Operations Research 18 (2010Mar), no. 1, 5–24.
P. E. Gill, G. H. Golub, W. Murray, and M. A. Saunders, Methods for modifying matrix factorizations, Math. Comp. 28 (1974), 505–535. MR 343558, DOI 10.1090/S0025-5718-1974-0343558-6
Gene H. Golub and Charles F. Van Loan, Matrix computations, 4th ed., Johns Hopkins Studies in the Mathematical Sciences, Johns Hopkins University Press, Baltimore, MD, 2013. MR 3024913
Ming Gu and Stanley C. Eisenstat, A stable and efficient algorithm for the rank-one modification of the symmetric eigenproblem, SIAM J. Matrix Anal. Appl. 15 (1994), no. 4, 1266–1276. MR 1293916, DOI 10.1137/S089547989223924X
P. Hall, D. Marshall, and R. Martin, Merging and splitting eigenspace models, IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (2000), no. 9, 1042–1049.
Nicholas J. Higham, Accuracy and stability of numerical algorithms, 2nd ed., Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2002. MR 1927606, DOI 10.1137/1.9780898718027
Marc Moonen, Paul Van Dooren, and Joos Vandewalle, A singular value decomposition updating algorithm for subspace tracking, SIAM J. Matrix Anal. Appl. 13 (1992), no. 4, 1015–1038. MR 1182710, DOI 10.1137/0613061
Benjamin Peherstorfer and Karen Willcox, Online adaptive model reduction for nonlinear systems via low-rank updates, SIAM J. Sci. Comput. 37 (2015), no. 4, A2123–A2150. MR 3384839, DOI 10.1137/140989169
G. W. Stewart, Matrix algorithms. Vol. I, Society for Industrial and Applied Mathematics, Philadelphia, PA, 1998. Basic decompositions. MR 1653546, DOI 10.1137/1.9781611971408
R. C. Thompson, The behavior of eigenvalues and singular values under perturbations of restricted rank, Linear Algebra Appl. 13 (1976), no. 1-2, 69–78. Collection of articles dedicated to Olga Taussky Todd. MR 407051, DOI 10.1016/0024-3795(76)90044-6
R. Zimmerman, Manifold interpolation and model reduction, arXiv:1902.06502v1 (2019).
Ralf Zimmermann, Benjamin Peherstorfer, and Karen Willcox, Geometric subspace updates with applications to online adaptive nonlinear model reduction, SIAM J. Matrix Anal. Appl. 39 (2018), no. 1, 234–261. MR 3763927, DOI 10.1137/17M1123286
R. Zimmermann and K. Willcox, An accelerated greedy missing point estimation procedure, SIAM J. Sci. Comput. 38 (2016), no. 5, A2827–A2850. MR 3544659, DOI 10.1137/15M1042899