Research on metaheuristics has focused almost exclusively on (novel) algorithmic development and on competitive testing, both of which have been frequently argued to yield very little generalizable knowledge. The main goal of this paper is to promote meta-analysis — a systematic statistical examination that combines the results of several independent studies —as a more suitable way to obtain problem- and implementation-independent insights on metaheuristics. Meta-analysis is widely used in several scientific domains, most notably the medical sciences (e.g., to establish the efficacy of a certain treatment). To the best of our knowledge this is the first meta-analysis in the field of metaheuristics. To illustrate the approach, we carry out a meta-analysis to gain insights into the importance of the adaptive layer in adaptive large neighborhood search (ALNS). Although ALNS has been widely used to solve a broad range of problems, it has not yet been established whether or not adaptiveness actually contributes to the performance of an ALNS algorithm. A total of 134 studies were identified through Google Scholar or personal email correspondence with researchers in the domain, 63 of which fit our eligibility criteria. After sending requests for data to the authors of the eligible studies, we obtained results for 25 different implementations of ALNS, which were analysed using a random-effects model. On average, the addition of an adaptive layer in an ALNS algorithm improves the objective function value by 0.14% (95% confidence interval 0.07 to 0.22%). Although the adaptive layer can (and in a limited number of studies does) have an added value, it also adds considerable complexity and can therefore only be recommended in some very specific situations. These findings underline the importance of evaluating the contribution of metaheuristic components, and of knowledge over competitive testing.

中文翻译：

---

元启发式的元分析：量化自适应大邻域搜索中自适应性的影响

元启发式的研究几乎完全集中在（新颖的）算法开发和竞争性测试上，这两者经常被争论产生很少的泛化知识。本文的主要目标是促进元分析——一种结合多项独立研究结果的系统统计检查——作为一种更合适的方式来获得与问题和实施无关的元启发式见解。荟萃分析广泛用于多个科学领域，最显着的是医学科学（例如，确定某种治疗的功效）。据我们所知，这是元启发式领域的第一个元分析。为了说明该方法，我们进行了元分析，以深入了解自适应层在自适应大邻域搜索 (ALNS) 中的重要性。尽管 ALNS 已被广泛用于解决范围广泛的问题，但尚未确定自适应性是否真的有助于 ALNS 算法的性能。通过 Google 学术搜索或与该领域研究人员的个人电子邮件通信，共确定了 134 项研究，其中 63 项符合我们的资格标准。在向符合条件的研究的作者发送数据请求后，我们获得了 25 种不同的 ALNS 实施结果，并使用随机效应模型对其进行了分析。平均而言，在 ALNS 算法中添加自适应层可将目标函数值提高 0.14%（95% 置信区间为 0.07 至 0.22%）。尽管自适应层可以（并且在有限数量的研究中确实如此）具有附加值，但它也增加了相当大的复杂性，因此只能在某些非常特定的情况下推荐使用。这些发现强调了评估元启发式组件的贡献以及知识对竞争测试的重要性。