Skip to main content
SpringerLink
Log in

Nonmonotone inexact restoration approach for minimization with orthogonality constraints

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

Minimizing a differentiable function restricted to the set of matrices with orthonormal columns finds applications in several fields of science and engineering. In this paper, we propose to solve this problem through a nonmonotone variation of the inexact restoration method consisting of two phases: restoration phase, aimed to improve feasibility, and minimization phase, aimed to decrease the function value in the tangent set of the constraints. For this, we give a suitable characterization of the tangent set of the orthogonality constraints, allowing us to (i) deal with the minimization phase efficiently and (ii) employ the Cayley transform to bring a point in the tangent set back to feasibility, leading to a SVD-free restoration phase. Based on previous global convergence results for the nonmonotone inexact restoration algorithm, it follows that any limit point of the sequence generated by the new algorithm is stationary. Moreover, numerical experiments on three different classes of the problem indicate that the proposed method is reliable and competitive with other recently developed methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. Available at http://www.math.ucla.edu/~wotaoyin/papers/feasible_method_matrix_manifold.html

  2. http://www.manopt.org

References

  1. Abrudan, T., Eriksson, J., Koivunen, V.: Steepest descent algorithms for optimization under unitary matrix constraint. IEEE Trans. Signal Process. 56(3), 1134–1147 (2008)

    Article  MathSciNet  Google Scholar 

  2. Absil, P.-A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7(3), 303–330 (2007)

    Article  MathSciNet  Google Scholar 

  3. Absil, P.A., Mahony, R., Sepulchre, R.: Optimization algorithms on matrix manifolds. Princeton University Press, Princeton (2008)

  4. Absil, P.A., Malick, J.: Projection-like retractions on matrix manifolds. SIAM J. Optim. 22(1), 135–158 (2012)

    Article  MathSciNet  Google Scholar 

  5. Arouxét, B., Echebest, N.E., Pilotta, E.A.: Inexact Restoration method for nonlinear optimization without derivatives. J. Comput. Appl. Math. 290(15), 26–43 (2015)

    Article  MathSciNet  Google Scholar 

  6. Barzilai, J., Borwein, J.M.: Two point step size gradient methods. IMA J. Numer. Anal. 8, 141–148 (1988)

    Article  MathSciNet  Google Scholar 

  7. Birgin, E.G., Bueno, L.F., Martínez, J.M.: Assessing the reability of general-purpose inexact restoration methods. J. Comput. Appl. Math. 282, 1–16 (2015)

    Article  MathSciNet  Google Scholar 

  8. Birgin, E.G., Martínez, J.M., Raydan, Marcos: Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Optim. 10(4), 1196–1211 (2000)

    Article  MathSciNet  Google Scholar 

  9. Birgin, E.G., Martínez, J.M.: Local convergence of an inexact-restoration method and numerical experiments. J. Optim. Theory Appl. 127(2), 229–247 (2005)

    Article  MathSciNet  Google Scholar 

  10. Bueno, L.F., Martínez, J.M.: On the complexity of an inexact restoration method for constrained optimization. SIAM J. Optim. 30(1), 80–101 (2020)

    Article  MathSciNet  Google Scholar 

  11. Boumal, N., Mishra, B., Absil, P.-A., Sepulchre, R.: MAnopt, a Matlab Toolbox for optimization on manifolds. J. Mach. Learn. Res. 15, 1455–1459 (2014)

    MATH  Google Scholar 

  12. Bueno, L.F., Friedlander, A., Martínez, J.M., Sobral, F.N.C.: Inexact restoration method for derivative-free optimization with smooth constraints. SIAM J. Optim. 23(2), 1189–1213 (2013)

    Article  MathSciNet  Google Scholar 

  13. Bueno, L.F., Haeser, G., Martínez, J.M.: A flexible inexact-restoration method for constrained optimization. J. Optim. Theory Appl. 165, 188–208 (2015)

    Article  MathSciNet  Google Scholar 

  14. Cancès, E., Chakir, R., Maday, Y.: Numerical analysis of nonlinear eigenvalue problems. J. Sci. Comput. 45, 90–117 (2010)

    Article  MathSciNet  Google Scholar 

  15. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MathSciNet  Google Scholar 

  16. Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998)

    Article  MathSciNet  Google Scholar 

  17. Francisco, J.B., Gonçalves, D.S., Bazán, F.S.V., Paredes, L.L.T.: Nonmonotone Inexact Restoration Method for nonlinear programming, Computational Optimization and Applications to appear. https://doi.org/10.1007/s10589-019-00129-2(2019)

  18. Francisco, J.B., Viloche Bazán, F.S.: Nonmonotone algorithm for minimization on closed sets with application to minimization on Stiefel manifolds. J. Comp. and Appl. Math. 236(10), 2717–2727 (2012)

    Article  MathSciNet  Google Scholar 

  19. Francisco, J.B., Bazán, F.S.V., Weber Mendonça, M.: Non-monotone algorithm for minimization on arbitrary domains with applications to large-scale orthogonal Procrustes problem, Appl. Num. Math. 112, 51–64 (2017)

    Article  Google Scholar 

  20. Francisco, J.B., Martínez, J.M., Martínez, L., Pisnitchenko, F.: Inexact restoration method for minimization problems arising in electronic structure calculations. Comput. Optim. Appl. 50, 555–590 (2011)

    Article  MathSciNet  Google Scholar 

  21. Fischer, A., Friedlander, A.: A new line search inexact restoration approach for nonlinear programming. Comput. Optim. Appl. 46, 333–346 (2010)

    Article  MathSciNet  Google Scholar 

  22. Golub, G.H., Van Loan, C.F.: Matrix Computations, 4th ed. The Johns Hopkins University Press (2013)

  23. Helgaker, T., JøRgensen, J., Olsen, J.: Electronic - Structure theory. Wiley, New York (2000)

  24. Janin, R.: Directional derivative of the marginal function in non linear programming. In: Sensitivity, Stability and Parametric Analysis, Math. Program. Stud., pp. 110–126. Springer, Berlin (1984)

  25. Jiang, B., Dai, Y.H.: A framework of constraint preserving update schemes for optimization on Stiefel manifold. Math. Program., Ser. A 153(2), 535–575 (2015)

    Article  MathSciNet  Google Scholar 

  26. Journée, M., Nesterov, Y., Richtárik, P., Sepulchre, R.: Generalized power method for sparse principal component analysis. J. Mach. Learn. Res. 11, 517–553 (2010)

    MathSciNet  MATH  Google Scholar 

  27. Manton, J.H.: Optimization algorithms exploiting unitary constraints. IEEE Trans. Signal Process. 50(3), 635–650 (2002)

    Article  MathSciNet  Google Scholar 

  28. Martínez, J.M., Pilotta, E.A.: Inexact restoration algorithm for constrained optimization. J. Optim. Theory App. 104(1), 135–163 (2000)

    Article  MathSciNet  Google Scholar 

  29. Martínez, J.M., Svaiter, B.F.: A practical optimality condition without constraint qualifications for nonlinear programming. J. Optimiz. Theory App. 118 (1), 117–133 (2003)

    Article  MathSciNet  Google Scholar 

  30. Nishimori, Y., Akaho, S.: Learning algorithms utilizing quasi-geodesic flows on the Stiefel manifold. Neurocomputing 67, 106–135 (2005)

    Article  Google Scholar 

  31. Shariff, M.: A constrained conjugate gradient method and the solution of linear equations. Comp. Mathem. Appl. 30(11), 25–37 (1995)

    Article  MathSciNet  Google Scholar 

  32. Karas, E. W., Pilotta, E., Ribeiro, A.: Numerical comparison of merit function with filter criterion in inexact restoration algorithms using hard-spheres problems. Comput. Optim. Appl. 44, 427–441 (2009)

    Article  MathSciNet  Google Scholar 

  33. Kaya, C.Y.: Inexact restoration for Runge–Kutta discretization of optimal control problems. SIAM J. Numer. Anal. 48(4), 1492–1517 (2010)

    Article  MathSciNet  Google Scholar 

  34. Kohn, W.: Nobel lecture: electronic structure of matter–wave functions and density functionals. Rev. Modern Phys. 71(5), 1253–1266 (1999)

    Article  Google Scholar 

  35. Krejić, N., Martínez, J.M.: Inexact restoration approach for minimization with inexact evaluation of the objective function. Mathematics of Computation 85, 1775–1791 (2016)

    Article  MathSciNet  Google Scholar 

  36. Wen, Z., Yin, W.: A feasible method for optimization with orthogonality constraints. Math. Program. Ser. A, 142, 397–434 (2013)

  37. Zhang, H., Hager, W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optimiz. 14(4), 1043–1056 (2004)

    Article  MathSciNet  Google Scholar 

  38. Zhao, Z., Bai, Z.-J., Jin, X.-Q.: A Riemannian Newton algorithm for nonlinear eigenvalue problems. SIAM J. Matrix Anal. Appl. 36(2), 752–774 (2015)

    Article  MathSciNet  Google Scholar 

  39. Zhu, X.: A feasible filter method for the nearest low-rank correlation matrix problem. Numer. Algorithm. 69, 763–784 (2015)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

D.S.G thanks CAPES/Print Processes 88881.310538/2018-01 and 88887.465828/2019-00 which allow him to present part of this work at ICCOPT 2019 and supported his period as visiting professor at École Polytechnique where part of this work was carried out. We also thank anonymous referees whose comments helped to improve the quality of this work.

Funding

J.B.F., D.S.G, and F.S.V.B are grateful to CNPq by the financial support (Grants n. 421386/2016-9 and 308523/2017-2). L.L.T.P would like to thank CAPES for the scholarship during her Ph.D. at Universidade Federal de Santa Catarina. This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.

Author information

Authors and Affiliations

  1. CFM, Universidade Federal de Santa Catarina, Florianópolis, Brazil

    Juliano B. Francisco, Douglas S. Gonçalves & Fermín S. V. Bazán

  2. FCM, Universidad Nacional Mayor de San Marcos, Lima, Peru

    Lila L. T. Paredes

Authors
  1. Juliano B. Francisco
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Douglas S. Gonçalves
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fermín S. V. Bazán
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lila L. T. Paredes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juliano B. Francisco.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix

A Tables of numerical results

This appendix gathers the tables with numerical results for the experiments discussed in Sections 6.3, 6.4, and 6.5. The symbol “–” means that the solver could not achieve \(\|\tilde {P}_{S_{k}} (\nabla f(Y_{k})) \| < \varepsilon = 10^{-6}\), using less than IT\(_{\max \limits }=2,000\) iterations and FE\(_{\max \limits }=4,000\) function evaluations.

Only for test problems of Table 7, the maximum number of iterations was increased to IT\(_{\max \limits }=5,000\) and function evaluations to IT\(_{\max \limits }=10,000\). Moreover, for this last case, we set M = 0 and \(\delta _{CG} = 10^{-4} \| P_{S_{0}} (Y_{0}) \|\) in IRNOM.

Table 1 CPU time (seconds) in random instances of LEP
Full size table
Table 2 CPU time (seconds) in random instances of NEP with α = 1
Full size table
Table 3 CPU time (seconds) in random instances of NEP with α = 10
Full size table
Table 4 CPU time (seconds) in random instances of NEP with α = 100
Full size table
Table 5 CPU time in random instances of OPP (uniformly distributed singular values)
Full size table
Table 6 CPU time in random instances of OPP (equally spaced singular values)
Full size table
Table 7 CPU time in random instances of OPP (clustered eigenvalues)
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Francisco, J.B., Gonçalves, D.S., Bazán, F.S.V. et al. Nonmonotone inexact restoration approach for minimization with orthogonality constraints. Numer Algor 86, 1651–1684 (2021). https://doi.org/10.1007/s11075-020-00948-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-020-00948-z

Keywords

Mathematics subject classification (2010)

Search

Navigation