Hyper-heuristic methodologies have been extensively and successfully used to generate combinatorial optimization heuristics. On the other hand, there have been almost no attempts to build a hyper-heuristic to evolve an algorithm for solving real-valued optimization problems. In our previous research, we succeeded to evolve a Nelder–Mead-like real function minimization heuristic using genetic programming and the primitives extracted from the original Nelder–Mead algorithm. The resulting heuristic was better than the original Nelder–Mead method in the number of solved test problems but it was slower in that it needed considerably more cost function evaluations to solve the problems also solved by the original method. In this paper we exploit grammatical evolution as a hyper-heuristic to evolve heuristics that outperform the original Nelder–Mead method in all aspects. However, the main goal of the paper is not to build yet another real function optimization algorithm but to shed some light on the influence of different factors on the behavior of the evolution process as well as on the quality of the obtained heuristics. In particular, we investigate through extensive evolution runs the influence of the shape and dimensionality of the training function, and the impact of the size limit set to the evolving algorithms. At the end of this research we succeeded to evolve a number of heuristics that solved more test problems and in fewer cost function evaluations than the original Nelder–Mead method. Our solvers are also highly competitive with the improvements made to the original method based on rigorous mathematical convergence proofs found in the literature. Even more importantly, we identified some directions in which to continue the work in order to be able to construct a productive hyper-heuristic capable of evolving real function optimization heuristics that would outperform a human designer in all aspects.

中文翻译：

---

语法演化作为一种​​超启发式来演化确定性实值优化算法

超启发式方法已被广泛并成功地用于生成组合优化启发式方法。另一方面，几乎没有人尝试构建超启发式算法来演化求解实值优化问题的算法。在我们之前的研究中，我们成功地使用遗传编程和从原始 Nelder-Mead 算法中提取的基元来演化出类似 Nelder-Mead 的实函数最小化启发式算法。由此产生的启发式方法在解决的测试问题数量上优于原始 Nelder-Mead 方法，但速度较慢，因为它需要更多的成本函数评估来解决原始方法也解决的问题。在本文中，我们利用语法进化作为一种​​超启发式方法来进化在所有方面都优于原始 Nelder-Mead 方法的启发式方法。然而，本文的主要目标不是构建另一个真正的函数优化算法，而是阐明不同因素对进化过程行为的影响以及对所获得启发式算法的质量的影响。特别是，我们通过广泛的进化研究了训练函数的形状和维度的影响，以及大小限制集对进化算法的影响。在这项研究结束时，我们成功地发展出许多启发式方法，与原始 Nelder-Mead 方法相比，这些方法可以解决更多的测试问题和更少的成本函数评估。我们的求解器在对基于文献中严格数学收敛证明的原始方法的改进方面也具有很强的竞争力。更重要的是，我们确定了一些继续工作的方向，以便能够构建一个高效的超启发式算法，能够进化出在各个方面都优于人类设计师的实际功能优化启发式算法。