Skip to main content
SpringerLink
Log in

Generalized Continuation Newton Methods and the Trust-Region Updating Strategy for the Underdetermined System

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

This paper considers the generalized continuation Newton method and the trust-region updating strategy for the underdetermined system of nonlinear equations. Moreover, in order to improve its computational efficiency, the new method will not update the Jacobian matrix when the current Jacobian matrix performs well. The numerical results show that the new method is more robust and faster than the traditional optimization method such as the Levenberg–Marquardt method (a variant of trust-region methods, the built-in subroutine fsolve.m of the MATLAB R2020a environment). The computational time of the new method is about 1/8 to 1/50 of that of fsolve. Furthermore, it also proves the global convergence and the local superlinear convergence of the new method under some standard assumptions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data Availability Statement

If it is requested, we will provide the test data. Code availability (software application or custom code) If it is requested, we will provide the code.

References

  1. Andrei, N.: An unconstrained optimization test functions collection. Environ. Sci. Technol. 10, 6552–6558 (2008)

    MathSciNet  Google Scholar 

  2. Allgower, E.L., Georg, K.: Introduction to Numerical Continuation Methods. SIAM, Philadelphia (2003)

    MATH  Google Scholar 

  3. Ascher, U.M., Petzold, L.R.: Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations. SIAM, Philadelphia (1998)

    MATH  Google Scholar 

  4. Axelsson, O., Sysala, S.: Continuation Newton methods. Comput. Math. Appl. 70, 2621–2637 (2015)

    MathSciNet  MATH  Google Scholar 

  5. Branin, F.H.: Widely convergent method for finding multiple solutions of simultaneous nonlinear equations. IBM J. Res. Dev. 16, 504–521 (1972)

    MathSciNet  MATH  Google Scholar 

  6. Brenan, K.E., Campbell, S.L., Petzold, L.R.: Numerical Solution of Initial-Value Problems in Differential-Algebraic Equations. SIAM, Philadelphia (1996)

    MATH  Google Scholar 

  7. Conn, A.R., Gould, N., Toint, Ph.L: Trust-Region Methods. SIAM, Philadelphia (2000)

  8. Davidenko, D.F.: On a new method of numerical solution of systems of nonlinear equations (in Russian). Dokl. Akad. Nauk SSSR 88, 601–602 (1953)

    Google Scholar 

  9. Deuflhard, P.: Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithms. Springer, Berlin (2004)

    MATH  Google Scholar 

  10. Deuflhard, P., Pesch, H.J., Rentrop, P.: A modified continuation method for the numerical solution of nonlinear two-point boundary value problems by shooting techniques. Numer. Math. 26, 327–343 (1975)

    MathSciNet  MATH  Google Scholar 

  11. Dennis, J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. SIAM, Philadelphia (1996)

    MATH  Google Scholar 

  12. Doedel, E.J.: Lecture notes in numerical analysis of nonlinear equations. In: Krauskopf, B., Osinga, H.M., Galán-Vioque, J. (eds.) Numerical Continuation Methods for Dynamical Systems, pp. 1–50. Springer, Berlin (2007)

    Google Scholar 

  13. Fan, J.-Y., Yuan, Y.-X.: On the quadratic convergence of the Levenberg–Marquardt method. Computing 74, 23–39 (2005)

    MathSciNet  MATH  Google Scholar 

  14. Gottlieb, S., Shu, C.W.: Total variation diminishing Runge–Kutta schemes. Math. Comput. 67, 73–85 (1998)

    MathSciNet  MATH  Google Scholar 

  15. Gottlieb, S., Shu, C.W., Tadmor, E.: Strong stability preserving high order time discretization methods. SIAM Rev. 43, 89–112 (2001)

    MathSciNet  MATH  Google Scholar 

  16. Golub, G.H., Van Loan, C.F.: Matrix Computation, 4th edn. The John Hopkins University Press, Baltimore (2013)

    MATH  Google Scholar 

  17. Griewank, A.: On solving nonlinear equations with simple singularities or nearly singular solutions. SIAM Rev. 27, 537–563 (1985)

    MathSciNet  MATH  Google Scholar 

  18. Hairer, E., Wanner, G.: Solving Ordinary Differential Equations II, Stiff and Differential-Algebraic Problems, 2nd edn. Springer, Berlin (1996)

    MATH  Google Scholar 

  19. Hiebert, K.L.: An evaluation of mathematical software that solves systems of nonlinear equations. ACM Trans. Math. Softw. 8, 5–20 (1982)

    MATH  Google Scholar 

  20. Higham, D.J.: Trust region algorithms and timestep selection. SIAM J. Numer. Anal. 37, 194–210 (1999)

    MathSciNet  MATH  Google Scholar 

  21. Izmailov, A.F., Solodov, M.V., Uskov, E.I.: A globally convergent Levenberg–Marquardt method for equality-constrained optimization. Comput. Optim. Appl. 72, 215–239 (2019)

    MathSciNet  MATH  Google Scholar 

  22. Kalaba, R.F., Zagustin, E., Holbrow, W., Huss, R.: A modification of Davidenko’s method for nonlinear systems. Comput. Math. Appl. 3, 315–319 (1977)

    MathSciNet  MATH  Google Scholar 

  23. Kelley, C.T., Keyes, D.E.: Convergence analysis of pseudo-transient continuation. SIAM J. Numer. Anal. 35, 508–523 (1998)

    MathSciNet  MATH  Google Scholar 

  24. Kelley, C.T.: Solving Nonlinear Equations with Newton’s Method. SIAM, Philadelphia (2003)

    MATH  Google Scholar 

  25. Kelley, C.T.: Numerical methods for nonlinear equations. Acta Numer. 27, 207–287 (2018)

    MathSciNet  MATH  Google Scholar 

  26. Liu, D.G., Fei, J.G.: Digital Simulation Algorithms for Dynamic Systems (in Chinese). Science Press, Beijing (2000)

    Google Scholar 

  27. Levenberg, K.: A method for the solution of certain problems in least squares. Q. Appl. Math. 2, 164–168 (1944)

    MathSciNet  MATH  Google Scholar 

  28. Liao, S.J.: Homotopy Analysis Method in Nonlinear Differential Equations. Springer, Berlin (2012)

    MATH  Google Scholar 

  29. Lukšan, L.: Inexact trust region method for large sparse systems of nonlinear equations. J. Optim. Theory Appl. 81, 569–590 (1994)

    MathSciNet  MATH  Google Scholar 

  30. Luo, X.-L., Liu, D.G.: Real-time simulation algorithms for computing differential-algebraic equation. Chin. J. Numer. Math. Appl. 22, 71–80 (2001)

    MathSciNet  Google Scholar 

  31. Luo, X.-L.: Singly diagonally implicit Runge–Kutta methods combining line search techniques for unconstrained optimization. J. Comput. Math. 23, 153–164 (2005)

    MathSciNet  MATH  Google Scholar 

  32. Luo, X.-L.: A trajectory-following method for solving the steady state of chemical reaction rate equations. J. Theor. Comput. Chem. 8, 1025–1044 (2009)

    Google Scholar 

  33. Luo, X.-L.: A second-order pseudo-transient method for steady-state problems. Appl. Math. Comput. 216, 1752–1762 (2010)

    MathSciNet  MATH  Google Scholar 

  34. Luo, X.-L., Liao, L.-Z., Tam, H.-W.: Convergence analysis of the Levenberg–Marquardt method. Optim. Methods Softw. 22, 659–678 (2007)

    MathSciNet  MATH  Google Scholar 

  35. Luo, X.-L., Lv, J.-H., Sun, G.: Continuation methods with the trusty time-stepping scheme for linearly constrained optimization with noisy data. Optim. Eng. (2021). https://doi.org/10.1007/s11081-020-09590-z

    Article  Google Scholar 

  36. Luo, X.-L., Xiao, H., Lv, J.-H.: Continuation Newton methods with the residual trust-region time-stepping scheme for nonlinear equations. Numer. Algorithms (2021). https://doi.org/10.1007/s11075-021-01112-x

    Article  Google Scholar 

  37. Luo, X.-L., Yao, Y.-Y.: Primal-dual path-following methods and the trust-region strategy for linear programming with noisy data. J. Comput. Math. (2021). https://doi.org/10.4208/jcm.2101-m2020-0173. arXiv:2006.07568

    Article  Google Scholar 

  38. Luo, X.-L., Xiao, H., Lv, J.-H., Zhang, S.: Explicit pseudo-transient continuation and the trust-region updating strategy for unconstrained optimization. Appl. Numer. Math. 165, 290–302 (2021)

    MathSciNet  MATH  Google Scholar 

  39. MATLAB v9.8.0 (R2020a): The MathWorks Inc. http://www.mathworks.com (2020)

  40. Marquardt, D.: An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math. 11, 431–441 (1963)

    MathSciNet  MATH  Google Scholar 

  41. Moré, J.J.: The Levenberg–Marquardt algorithm: implementation and theory. In: Watson, G.A. (ed.) Numerical Analysis, Lecture Notes in Mathematics, vol. 630, pp. 105–116. Springer, Berlin (1978)

    Google Scholar 

  42. Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981)

    MathSciNet  MATH  Google Scholar 

  43. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, Berlin (1999)

    MATH  Google Scholar 

  44. Ortega, J.M., Rheinboldt, W.C.: Iteration Solution of Nonlinear Equations in Several Variables. SIAM, Philadelphia (2000)

    MATH  Google Scholar 

  45. Powell, M.J.D.: Convergence properties of a class of minimization algorithms. In: Mangasarian, O.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear Programming, vol. 2, pp. 1–27. Academic Press, New York (1975)

    Google Scholar 

  46. Qian, J., Andrew, A.L., Chu, D.L., Tan, R.C.E.: Methods for solving underdetermined systems. Numer. Linear Algebra Appl. 25, e2127 (2017)

    MathSciNet  MATH  Google Scholar 

  47. Surjanovic, S., Bingham, D.: Virtual library of simulation experiments: test functions and datasets. http://www.sfu.ca/~ssurjano (2020)

  48. Shampine, L.F., Gladwell, I., Thompson, S.: Solving ODEs with MATLAB. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  49. Sun, W.Y., Yuan, Y.X.: Optimization Theory and Methods: Nonlinear Programming. Springer, New York (2006)

    MATH  Google Scholar 

  50. Tanabe, K.: Continuous Newton–Raphson method for solving an underdetermined system of nonlinear equations. Nonlinear Anal. 3, 495–503 (1979)

    MathSciNet  MATH  Google Scholar 

  51. Watson, L.T., Sosonkina, M., Melville, R.C., Morgan, A.P., Walker, H.F.: HOMPACK90: A suite of fortran 90 codes for globally convergent homotopy algorithms. ACM Trans. Math. Softw. 23, 514–549 (1997)

    MathSciNet  MATH  Google Scholar 

  52. Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg–Marquardt method. Computing 15(Suppl.), 239–249 (2001)

    MathSciNet  MATH  Google Scholar 

  53. Yuan, Y.X.: Trust region algorithms for nonlinear equations. Information 1, 7–20 (1998)

    MathSciNet  MATH  Google Scholar 

  54. Yuan, Y.X.: Recent advances in trust region algorithms. Math. Program. 151, 249–281 (2015)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors are grateful to two anonymous referees for their comments and suggestions which greatly improve presentation of this paper.

Funding

This work was supported in part by Grant 61876199 from National Natural Science Foundation of China, and Grant YJCB2011003HI from the Innovation Research Program of Huawei Technologies Co., Ltd.

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Beijing University of Posts and Telecommunications, P. O. Box 101, Xitucheng Road No. 10, Haidian District, Beijing, 100876, China

    Xin-long Luo & Hang Xiao

Authors
  1. Xin-long Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hang Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin-long Luo.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Luo, Xl., Xiao, H. Generalized Continuation Newton Methods and the Trust-Region Updating Strategy for the Underdetermined System. J Sci Comput 88, 56 (2021). https://doi.org/10.1007/s10915-021-01566-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-021-01566-0

Keywords

Mathematics Subject Classification

Search

Navigation