Abstract
The fast-growing need for grey-box and black-box optimization methods for constrained global optimization problems in fields such as medicine, chemistry, engineering and artificial intelligence, has led to the development of new efficient algorithms for finding the best possible solution. In this work, we present DEFT-FUNNEL, an open-source global optimization algorithm for general constrained grey-box and black-box problems that belongs to the class of trust-region sequential quadratic optimization algorithms. Polynomial interpolation models are used as surrogates for the black-box functions and a clustering-based multistart strategy is applied for searching for the global minima. Numerical experiments show that DEFT-FUNNEL compares favorably with state-of-the-art methods on two sets of benchmark problems: one set containing problems where every function is a black box and another set with problems where some of the functions and their derivatives are known to the solver. The code as well as the test sets used for experiments are available at the Github repository http://github.com/phrsampaio/deft-funnel.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Amaioua N, Audet C, Conn AR, Digabel SL (2018) Efficient solution of quadratically constrained quadratic subproblems within the mesh adaptive direct search algorithm. Eur J Oper Res 268(1):13–24. https://doi.org/10.1016/j.ejor.2017.10.058. Retrieved from http://www.sciencedirect.com/science/article/pii/S0377221717309876
Armstrong JC, Favorite JA (2016) Using a derivative-free optimization method for multiple solutions of inverse transport problems. Optim Eng 17(1):105–125. https://doi.org/10.1007/s11081-015-9306-x
Audet C, Dennis J (2006) Mesh adaptive direct search algorithms for constrained optimization. SIAM J Optim 17(1):188–217. https://doi.org/10.1137/040603371
Audet C, Digabel SL, Peyrega M (2015) Linear equalities in blackbox optimization. Comput Optim Appl 61(1):1–23. https://doi.org/10.1007/s10589-014-9708-2
Audet C, Hare W (2017) Derivative-free and blackbox optimization. Springer, Cham. https://doi.org/10.1007/978-3-319-68913-5
Bajaj I, Iyer SS, Hasan MMF (2018) A trust region-based two phase algorithm for constrained black-box and grey-box optimization with infeasible initial point. Comput Chem Eng 116:306–321. https://doi.org/10.1016/j.compchemeng.2017.12.011, Retrieved from http://www.sciencedirect.com/science/article/pii/S0098135417304404
Beightler CS, Phillips DT (1976) Applied geometric programming. Wiley, New York
Birgin EG, Martínez JM, Raydan M (2000) Nonmonotone spectral projected gradient methods on convex sets. SIAM J Optim 10(4):1196–1211
Boukouvala F, Hasan MMF, Floudas CA (2017) Global optimization of general constrained grey-box models: new method and its application to constrained pdes for pressure swing adsorption. J Global Optim 67(1):3–42. https://doi.org/10.1007/s10898-015-0376-2
Bueno L, Friedlander A, Martínez J, Sobral F (2013) Inexact restoration method for derivative-free optimization with smooth constraints. SIAM J Optim 23(2):1189–1213. https://doi.org/10.1137/110856253
Coello CAC, Montes EM (2002) Constraint-handling in genetic algorithms through the use of dominance-based tournament selection. Adv Eng Inf 16(3):193–203. https://doi.org/10.1016/S1474-0346(02)00011-3. Retrieved from http://www.sciencedirect.com/science/article/pii/S1474034602000113
Conn AR, Scheinberg K, Vicente LN (2009) Introduction to derivative-free optimization. MPS-SIAM Book Series on Optimization, Philadelphia
Deb K (2000) An efficient constraint handling method for genetic algorithms. Comput Methods Appl Mech Eng 186(2):311–338. https://doi.org/10.1016/S0045-7825(99)00389-8. Retrieved from http://www.sciencedirect.com/science/article/pii/S0045782599003898
Digabel SL, Wild SM (2015) A taxonomy of constraints in simulation-based optimization. arXiv:1505.07881
Echebest N, Schuverdt ML, Vignau RP (2017) An inexact restoration derivative-free filter method for nonlinear programming. Comput Appl Math 36(1):693–718. https://doi.org/10.1007/s40314-015-0253-0
Floudas C (2000) Deterministic global optimization: theory, methods and applications. Springer, Boston, MA. https://doi.org/10.1007/978-1-4757-4949-6
Floudas C, Pardalos P, Adjiman C, Esposito WR, Gümüs ZH, Harding, ST, Schweiger CA (1999) Handbook of test problems in local and global optimization. Springer US. https://doi.org/10.1007/978-1-4757-3040-1
Floudas CA, Pardalos PM (1990) A collection of test problems for constrained global optimization algorithms. Springer, Berlin
Gómez S, Levy AV (1982) The tunnelling method for solving the constrained global optimization problem with several non-connected feasible regions. In: Hennart JP (ed) Numerical analysis. Springer, Heidelberg, pp 34–47
Gould NIM, Toint PL (2010) Nonlinear programming without a penalty function or a filter. Math Program 122(1):155–196. https://doi.org/10.1007/s10107-008-0244-7
Gratton S, Toint PL, Tröltzsch A (2011) An active-set trust-region method for bound-constrained nonlinear optimization without derivatives. Optim Methods Softw 26(4–5):875–896
Griewank A (2003) A mathematical view of automatic differentiation. Acta Numerica 12:321–398. https://doi.org/10.1017/S0962492902000132
Griewank A, Walther A (2008) Evaluating derivatives (Second ed), Society for Industrial and Applied Mathematics. Retrieved from https://epubs.siam.org/doi/abs/10.1137/1.9780898717761. https://doi.org/10.1137/1.9780898717761
Hesse R (1973) A heuristic search procedure for estimating a global solution of nonconvex programming problems. Oper Res 21(6):1267–1280. https://doi.org/10.1287/opre.21.6.1267
Hock W, Schittkowski K (1980) Test examples for nonlinear programming codes. J Optim Theory Appl 30(1):127–129. https://doi.org/10.1007/BF00934594
Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Global Optim 13(4):455–492. https://doi.org/10.1023/A:1008306431147
Kan AHGR, Timmer GT (1987a) Stochastic global optimization methods part i: Clustering methods. Math Program 39(1):27–56. https://doi.org/10.1007/BF02592070
Kan AHGR, Timmer GT (1987b) Stochastic global optimization methods part ii: multi level methods. Math Program 39(1):57–78. https://doi.org/10.1007/BF02592071
Larson J, Menickelly M, Wild SM (2019) Derivative-free optimization methods. Acta Numer 28:287–404. https://doi.org/10.1017/S0962492919000060
Le Digabel S (2011) Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans Math Softw 37(4):1–15
Lewis R, Torczon V (2002) A globally convergent augmented lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J Optim 12(4):1075–1089. https://doi.org/10.1137/S1052623498339727
Locatelli M (1998) Relaxing the assumptions of the multilevel single linkage algorithm. J Global Optim 13(1):25–42. https://doi.org/10.1023/A:1008246031222
Maratos N (1978) Exact penalty function algorithms for finite dimensional and control optimization problems., Department of Control Theory, Imperial College London. Retrieved from https://books.google.fr/books?id=Ar2AtgAACAAJ
MATLAB (2015b) Natick, Massachusetts, The MathWorks Inc
McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245. Retrieved from http://www.jstor.org/stable/1268522
Michalewicz Z, Schoenauer M (1996) Evolutionary algorithms for constrained parameter optimization problems. Evol Comput 4(1):1–32. https://doi.org/10.1162/evco.1996.4.1.1
Nocedal J, Wright SJ (2006) Numerical optimization, 2nd edn. Springer, New York
Powell MJD (1994) A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Gomez S, Hennart JP (eds), Advances in optimization and numerical analysis (pp. 51–67). Springer, Dordrecht. https://doi.org/10.1007/978-94-015-8330-5_4
Regis RG (2011) Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions. Comput Oper Res 38(5):837–853. https://doi.org/10.1016/j.cor.2010.09.013. Retrieved from http://www.sciencedirect.com/science/article/pii/S030505481000208X
Regis RG (2014) Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points. Eng Optim 46(2):218–243. https://doi.org/10.1080/0305215X.2013.765000
Regis RG, Shoemaker CA (2005) Constrained global optimization of expensive black box functions using radial basis functions. J Global Optim 31(1):153–171. https://doi.org/10.1007/s10898-004-0570-0
Regis RG, Shoemaker CA (2007) A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J Comput 19(4):497–509. https://doi.org/10.1287/ijoc.1060.0182
Rios LM, Sahinidis NV (2013) Derivative-free optimization: a review of algorithms and comparison of software implementations. J Global Optim 56(3):1247–1293. https://doi.org/10.1007/s10898-012-9951-y
Sampaio PR, Toint PL (2015) A derivative-free trust-funnel method for equality-constrained nonlinear optimization. Comput Optim Appl 61(1):25–49. https://doi.org/10.1007/s10589-014-9715-3
Sampaio PR, Toint PL (2016) Numerical experience with a derivative-free trust-funnel method for nonlinear optimization problems with general nonlinear constraints. Optim Methods Softw 31(3):511–534. https://doi.org/10.1080/10556788.2015.1135919
Scheinberg K, Toint PL (2010) Self-correcting geometry in model-based algorithms for derivative-free unconstrained optimization. SIAM J Optim 20(6):3512–3532
Sendín JOH, Banga JR, Csendes T (2009) Extensions of a multistart clustering algorithm for constrained global optimization problems. Ind Eng Chem Res 48(6):3014–3023. https://doi.org/10.1021/ie800319m
Zhang H, Conn A, Scheinberg K (2010) A derivative-free algorithm for least-squares minimization. SIAM J Optim 20(6):3555–3576. https://doi.org/10.1137/09075531X
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Ernesto G. Birgin.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sampaio, P.R. DEFT-FUNNEL: an open-source global optimization solver for constrained grey-box and black-box problems. Comp. Appl. Math. 40, 176 (2021). https://doi.org/10.1007/s40314-021-01562-y
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s40314-021-01562-y
Keywords
- Global optimization
- Constrained nonlinear optimization
- Black-box optimization
- Grey-box optimization
- Derivative-free optimization
- Simulation-based optimization