When developing optimisation algorithms, the focus often lies on obtaining an algorithm that is able to outperform other existing algorithms for some performance measure. It is not common practice to question the reasons for possible performance differences observed. These types of questions relate to evaluating the impact of the various heuristic parameters and often remain unanswered. In this paper, the focus is on gaining insight in the behaviour of a heuristic algorithm by investigating how the various elements operating within the algorithm correlate with performance, obtaining indications of which combinations work well and which do not, and how all these effects are influenced by the specific problem instance the algorithm is solving. We consider two approaches for analysing algorithm parameters and components—functional analysis of variance and multilevel regression analysis—and study the benefits of using both approaches jointly. We present the results of a combined methodology that is able to provide more insights than when the two approaches are used separately. The illustrative case studies in this paper analyse a large neighbourhood search algorithm applied to the vehicle routing problem with time windows and an iterated local search algorithm for the unrelated parallel machine scheduling problem with sequence-dependent setup times.

中文翻译：

---

分析启发式算法的组合方法

在开发优化算法时，通常的重点是获得在某些性能指标上能够胜过其他现有算法的算法。质疑观察到的性能差异的原因并不常见。这些类型的问题与评估各种启发式参数的影响有关，并且常常无法回答。在本文中，重点是通过研究算法中操作的各个元素如何与性能相关联，获得关于哪些组合有效，哪些无效以及如何影响所有这些影响的指示，从而获得启发式算法行为的见解。根据算法正在解决的特定问题实例。我们考虑两种用于分析算法参数和组成部分的方法-方差函数分析和多级回归分析-并共同研究使用这两种方法的好处。我们提供了一种组合方法的结果，该方法能够提供比分别使用两种方法时更多的见解。本文中的示例性案例研究分析了一种适用于带有时间窗的车辆路径问题的大型邻域搜索算法，以及一种针对不依赖序列的建立时间的并行机器调度问题的迭代局部搜索算法。