Instance Space Analysis (ISA) is a recently developed methodology to (a) support objective testing of algorithms and (b) assess the diversity of test instances. Representing test instances as feature vectors, the ISA methodology extends Rice’s 1976 Algorithm Selection Problem framework to enable visualization of the entire space of possible test instances, and gain insights into how algorithm performance is affected by instance properties. Rather than reporting algorithm performance on average across a chosen set of test problems, as is standard practice, the ISA methodology offers a more nuanced understanding of the unique strengths and weaknesses of algorithms across different regions of the instance space that may otherwise be hidden on average. It also facilitates objective assessment of any bias in the chosen test instances and provides guidance about the adequacy of benchmark test suites. This article is a comprehensive tutorial on the ISA methodology that has been evolving over several years, and includes details of all algorithms and software tools that are enabling its worldwide adoption in many disciplines. A case study comparing algorithms for university timetabling is presented to illustrate the methodology and tools.

实例空间分析 (ISA) 是最近开发的一种方法，用于 (a) 支持算法的客观测试和 (b) 评估测试实例的多样性。ISA 方法将测试实例表示为特征向量，它扩展了莱斯 1976 年的算法选择问题框架，使可能的测试实例的整个空间可视化，并深入了解实例属性如何影响算法性能。ISA 方法不是像标准做法那样报告一组选定测试问题的平均算法性能，而是提供对实例空间不同区域算法的独特优势和劣势的更细致的理解，否则这些优势和劣势可能被平均隐藏. 它还有助于客观评估所选测试实例中的任何偏差，并提供有关基准测试套件是否充分的指导。本文是关于 ISA 方法论的综合教程，该方法论已经发展了数年，包括所有算法和软件工具的详细信息，这些算法和软件工具使它能够在许多学科中得到全球采用。提出了一个比较大学时间表算法的案例研究，以说明方法和工具。