This paper systematically derives design dimensions for the structured evaluation of explainable artificial intelligence (XAI) approaches. These dimensions enable a descriptive characterization, facilitating comparisons between different study designs. They further structure the design space of XAI, converging towards a precise terminology required for a rigorous study of XAI. Our literature review differentiates between comparative studies and application papers, revealing methodological differences between the fields of machine learning, human-computer interaction, and visual analytics. Generally, each of these disciplines targets specific parts of the XAI process. Bridging the resulting gaps enables a holistic evaluation of XAI in real-world scenarios, as proposed by our conceptual model characterizing bias sources and trust-building. Furthermore, we identify and discuss the potential for future work based on observed research gaps that should lead to better coverage of the proposed model.

中文翻译：

---

我们应该相信 (X)AI 吗？结构化实验评估的设计尺寸

本文系统地推导出了可解释人工智能 (XAI) 方法的结构化评估的设计维度。这些维度能够进行描述性表征，促进不同研究设计之间的比较。它们进一步构建了 XAI 的设计空间，收敛于严格研究 XAI 所需的精确术语。我们的文献综述区分了比较研究和应用论文，揭示了机器学习、人机交互和视觉分析领域之间的方法论差异。通常，这些学科中的每一个都针对 XAI 过程的特定部分。正如我们表征偏见来源和建立信任的概念模型所提出的那样，弥合由此产生的差距可以在现实世界的场景中对 XAI 进行整体评估。