Skip to main content
SpringerLink
Log in

A derivative-free algorithm for spherically constrained optimization

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

Spherically constrained optimization, which minimizes an objective function on a unit sphere, has wide applications in numerical multilinear algebra, signal processing, solid mechanics, etc. In this paper, we consider a certain case that the derivatives of the objective function are unavailable. This case arises frequently in computational science, chemistry, physics, and other enormous areas. To explore the spherical structure of the above problem, we apply the Cayley transform to preserve iterates on the sphere and propose a derivative-free algorithm, which employs a simple model-based trust-region framework. Under mild conditions, global convergence of the proposed algorithm is established. Preliminary numerical experiments illustrate the promising performances of our algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

References

  1. Audet, C., Hare, W.: Derivative-Free and Blackbox Optimization. Springer, Berlin (2017)

    Book  Google Scholar 

  2. Boufounos, P., Baraniuk, R.: 1-bit compressive sensing. In: Conference on Information Sciences and Systems (CISS). Princeton (2008)

  3. Chang, J., Chen, Y., Qi, L.: Computing eigenvalues of large scale sparse tensors arising from a hypergraph. SIAM J. Sci. Comput. 38, A3618–A3643 (2016)

    Article  MathSciNet  Google Scholar 

  4. Chen, Y., Qi, L., Wang, Q.: Computing extreme eigenvalues of large scale Hankel tensors. J. Sci. Comput. 68, 716–738 (2016)

    Article  MathSciNet  Google Scholar 

  5. Chen, Y., Qi, L., Zhang, X.: The Fiedler vector of a Laplacian tensor for hypergraph partitioning. SIAM J. Sci. Comput. 39, A2508–A2537 (2017)

    Article  MathSciNet  Google Scholar 

  6. Conejo, P., Karas, E., Pedroso, L.: A trust-region derivative-free algorithm for constrained optimization. Optim. Methods Softw. 30, 1126–1145 (2015)

    Article  MathSciNet  Google Scholar 

  7. Conn, A.R., Scheinberg, K., Toint, PhL: On the convergence of derivative-free methods for unconstrained optimization. In: Iserles, A., Buhmann, M. (eds.) Approximation Theory and Optimization, Tributes to M. J. D. Powell, pp. 83–108. Cambridge University Press, Cambridge (1997)

    MATH  Google Scholar 

  8. Conn, A.R., Scheinberg, K., Toint, Ph.L.: A derivative free optimization algorithm in practice. In: Proceedings of the 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, pp. 1–11. AIAA, Reston, VA (1998)

  9. Conn, A.R., Scheinberg, K., Vicente, L.N.: Geometry of interpolation sets in derivative free optimization. Math. Program. 111, 141–172 (2008)

    Article  MathSciNet  Google Scholar 

  10. Conn, A.R., Scheinberg, K., Vicente, L.N.: Global convergence of general derivative-free trust-region algorithms to first- and second-order critical points. SIAM J. Optim. 20, 387–415 (2009)

    Article  MathSciNet  Google Scholar 

  11. Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to derivative-free optimization. MPS/SIAM Ser. Optim. 8. SIAM, Philadelphia (2009)

  12. Conn, A.R., Toint, Ph L.: An algorithm using quadratic interpolation for unconstrained derivative-free optimization. In: Pillo, G.Di, Giannessi, F. (eds.) Nonlinear Optimization and Application, pp. 27–47. Plenium Publishing, New York (1996)

    Chapter  Google Scholar 

  13. Dennis, J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Englewood Cliffs (1983)

    MATH  Google Scholar 

  14. Golub, G.H., Van Loan, C.F.: Matrix Computations. Johns Hopkins University Press, Baltimore (2013)

    MATH  Google Scholar 

  15. Gray, G., Kolda, T., Sale, K., Young, M.: Optimizing an empirical scoring function for transmembrane protein structure determination. INFORMS J. Comput. 16, 406–418 (2004)

    Article  MathSciNet  Google Scholar 

  16. Griewank, A.: Computational differentiation and optimization. In: Birge, J.R., Murty, K.G. (eds.) Mathematical Programming: State of the Art, pp. 102–131. The University of Michigan, Ann Arbor (1994)

    Google Scholar 

  17. Griewank, A., Coriiss, G.: Automatic Differentiation of Algorithms. SIAM, Philadelphia (1991)

    Google Scholar 

  18. Han, D.H., Dai, H., Qi, L.: Conditions for strong ellipticity of anisotropic elastic materials. J. Elast. 97, 1–13 (2009)

    Article  MathSciNet  Google Scholar 

  19. Hooke, R., Jeeves, T.A.: Direct search solution of numerical and statistical problems. J. Assoc. Comput. Mach. 8, 212–229 (1961)

    Article  Google Scholar 

  20. Huang, X.J., Zhu, D.T.: An interior affine scaling cubic regularization algorithm for derivative-free optimization subject to bound constraints. J. Comput. Appl. Math. 321, 108–127 (2017)

    Article  MathSciNet  Google Scholar 

  21. Laska, J.N., Wen, Z., Yin, W., Baraniuk, R.G.: Trust, but verify: fast and accurate signal recovering from 1-bit compressive measurements. Technical report. Rice University (2010)

  22. Liuzzi, G., Lucidi, S., Sciandrone, M.: Sequential penalty derivative-free methods for nonlinear constrained optimization. SIAM J. Optim. 20, 2614–2635 (2010)

    Article  MathSciNet  Google Scholar 

  23. Marazzi, M., Nocedal, J.: Wedge trust region methods for derivative-free optimization. Math. Program. 91, 289–305 (2002)

    Article  MathSciNet  Google Scholar 

  24. Meza, J.C., Martinez, M.L.: On the use of direct search methods for the molecular conformation problem. J. Comput. Chem. 15, 627–632 (1994)

    Article  Google Scholar 

  25. More, J.J., Wild, S.M.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20, 172–191 (2009)

    Article  MathSciNet  Google Scholar 

  26. Nazareth, L., Tseng, P.: Gilding the lily: a variant of the Nelder–Mead algorithm based on golden-section search. Comput. Optim. Appl. 22, 133–144 (2002)

    Article  MathSciNet  Google Scholar 

  27. Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7, 308–313 (1965)

    Article  MathSciNet  Google Scholar 

  28. Norman Katz, I., Cooper, L.: Optimal location on a sphere. Comput. Maths. Appls. 6, 175–196 (1980)

    Article  MathSciNet  Google Scholar 

  29. Oeuvray, R., Bierlaire, M.: A new derivative-free algorithm for the medical image registration problem. Int. J. Model. Simul. 27, 115–124 (2007)

    Article  Google Scholar 

  30. Powell, M.J.D.: A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Gomez, S., Hennart, J.-P. (eds.) Advances in Optimization and Numerical Analysis, pp. 51–67. Kluwer, Dordrecht (1994)

    Chapter  Google Scholar 

  31. Powell, M.J.D.: A direct search optimization method that models the objective by quadratic interpolation. In: Presentation at the 5th Stockholm Optimization Days (1994)

  32. Powell, M.J.D.: UOBYQA: unconsrained optimization by quadratic approximation. Math. Program. 92, 555–582 (2002)

    Article  MathSciNet  Google Scholar 

  33. Powell, M.J.D.: Least Frobenius norm updating of quadratic models that satisfy interpolation conditions. Math. Program. Ser. B 100, 183–215 (2004)

    Article  MathSciNet  Google Scholar 

  34. Powell, M.J.D.: The NEWUOA software for unconstrained optimization without derivatives. In: Di Pillo, G., Roma, M. (eds.) Large-Scale Nonlinear Optimization, pp. 255–297. Springer, New York (2006)

    Chapter  Google Scholar 

  35. Price, C.J., Coope, I.D., Byatt, D.: A convergent variant of the Nelder Mead algorithm. J. Optim. Theory Appl. 113, 5–19 (2002)

    Article  MathSciNet  Google Scholar 

  36. Qi, L.: Eigenvalues of a real supersymmetric tensor. J. Symbol. Comput. 40, 1302–1324 (2005)

    Article  MathSciNet  Google Scholar 

  37. Ragonneau, T., Zhang, Z.: PDFO: Powell’s derivative-free optimization software. Available at http://zhangzk.net/software.html (2019)

  38. Ryan, S.D., Zheng, X., Palffy-Muhoray, P.: Curvature driven foam coarsening on a sphere: a computer simulation. Phys. Rev. E 93, 053301–053301 (2016)

    Article  Google Scholar 

  39. Sampaio, P.R., Toint, PhL: A derivative-free trust-funnel method for equality-constrained. Comput. Optim. Appl. 61, 25–49 (2015)

    Article  MathSciNet  Google Scholar 

  40. Sun, W.Y., Yuan, Y.-X.: Optimization Theory and Methods. Vol. 1 of Springer Optimization and Its Applications. Springer, New York (2006)

    Google Scholar 

  41. Torczon, V.J.: Multi-directional search: a direct search algorithm for parallel machines. Ph.D. thesis, Rice University, Houston (1989)

  42. Torczon, V.J.: On the convergence of pattern search algorithms. SIAM J. Optim. 7, 1–25 (1997)

    Article  MathSciNet  Google Scholar 

  43. Wen, Z., Yin, W.: A feasible method for optimization with orthogonality constraints. Math. Program. 142, 397–434 (2013)

    Article  MathSciNet  Google Scholar 

  44. Winfield, D.: Function and Functional Optimization by Interpolation in Data Table. Ph.D. Thesis, Harvard University, Cambridge (1969)

  45. Winfield, D.: Function minimization by interpolation in a data table. J. Inst. Math. Appl. 12, 339–347 (1973)

    Article  MathSciNet  Google Scholar 

  46. Yamaguchi, K.: Borda winner in facility location problems on sphere. Soc. Choice Welf. 46, 893–898 (2016)

    Article  MathSciNet  Google Scholar 

  47. Zhang, H.C., Conn, A.R., Scheinberg, K.: A derivative-free algorithm for least-squares minimization. SIAM J. Optim. 20, 3555–3576 (2010)

    Article  MathSciNet  Google Scholar 

  48. Zhang, Z.: Sobolev seminorm of quadratic functions with applications to derivative-free optimization. Math. Program. 146, 77–96 (2014)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We thank Dr. Zaikun Zhang for his help on the COBYLA software and two referees for their valuable comments.

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Jiangsu Key Laboratory for NSLSCS, Nanjing Normal University, Nanjing, 210023, China

    Min Xi, Wenyu Sun & Hailin Sun

  2. School of Mathematics and Statistics, Guangdong University of Foreign Studies, Guangzhou, 510006, China

    Min Xi

  3. School of Mathematical Sciences, South China Normal University, Guangzhou, 510631, China

    Yannan Chen

Authors
  1. Min Xi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenyu Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yannan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hailin Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Wenyu Sun or Yannan Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is supported by the National Natural Science Foundation of China under Project Nos. 11571178, 11771405 and 11871276.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xi, M., Sun, W., Chen, Y. et al. A derivative-free algorithm for spherically constrained optimization. J Glob Optim 76, 841–861 (2020). https://doi.org/10.1007/s10898-020-00875-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-020-00875-2

Keywords

Search

Navigation