Abstract Evaluating local explanation methods is a difficult task due to the lack of a shared and universally accepted definition of explanation. In the literature, one of the most common ways to assess the performance of an explanation method is to measure the fidelity of the explanation with respect to the classification of a black box model adopted by an Artificial Intelligent system for making a decision. However, this kind of evaluation only measures the degree of adherence of the local explainer in reproducing the behavior of the black box classifier with respect to the final decision. Therefore, the explanation provided by the local explainer could be different in the content even though it leads to the same decision of the AI system. In this paper, we propose an approach that allows to measure to which extent the explanations returned by local explanation methods are correct with respect to a synthetic ground truth explanation. Indeed, the proposed methodology enables the generation of synthetic transparent classifiers for which the reason for the decision taken, i.e., a synthetic ground truth explanation, is available by design. Experimental results show how the proposed approach allows to easily evaluate local explanations on the ground truth and to characterize the quality of local explanation methods.

中文翻译：

---

评估基于真实情况的局部解释方法

摘要 由于缺乏共享和普遍接受的解释定义，评估本地解释方法是一项艰巨的任务。在文献中，评估解释方法性能的最常见方法之一是衡量解释相对于人工智能系统采用的黑盒模型分类的保真度来做出决策。然而，这种评估仅衡量本地解释器在再现黑盒分类器对最终决策的行为的遵守程度。因此，本地解释器提供的解释可能在内容上有所不同，即使它导致人工智能系统的相同决策。在本文中，我们提出了一种方法，该方法允许测量本地解释方法返回的解释在多大程度上是正确的，相对于合成的基本事实解释。事实上，所提出的方法能够生成合成透明分类器，其做出决定的原因，即合成的基本事实解释，可以通过设计获得。实验结果表明，所提出的方法如何能够轻松地评估对基本事实的局部解释，并表征局部解释方法的质量。可按设计提供。实验结果表明，所提出的方法如何能够轻松地评估对基本事实的局部解释，并表征局部解释方法的质量。可按设计提供。实验结果表明，所提出的方法如何能够轻松地评估对基本事实的局部解释，并表征局部解释方法的质量。