In this paper, we propose a novel explanation module to explain the predictions made by a deep network. The explanation module works by embedding a high-dimensional deep network layer nonlinearly into a low-dimensional explanation space while retaining faithfulness, so that the original deep learning predictions can be constructed from the few concepts extracted by the explanation module. We then visualize such concepts for human to learn about the high-level concepts that deep learning is using to make decisions. We propose an algorithm called Sparse Reconstruction Autoencoder (SRAE) for learning the embedding to the explanation space. SRAE aims to reconstruct part of the original feature space while retaining faithfulness. A pull-away term is applied to SRAE to make the explanation space more orthogonal. A visualization system is then introduced for human understanding of the features in the explanation space. The proposed method is applied to explain CNN models in image classification tasks, and several novel metrics are introduced to evaluate the performance of explanations quantitatively without human involvement. Experiments show that the proposed approach generates interesting explanations of the mechanisms CNN use for making predictions.

中文翻译：

---

将深层网络嵌入到视觉解释中

在本文中，我们提出了一种新颖的解释模块来解释深度网络所做的预测。解释模块的工作原理是将高维深度网络层非线性地嵌入到低维解释空间中，同时保持忠实性，从而可以从解释模块提取的少数概念中构建原始的深度学习预测。然后，我们将这些概念可视化以供人类了解深度学习用于做出决策的高级概念。我们提出了一种称为 Sparse Reconstruction Autoencoder (SRAE) 的算法，用于学习对解释空间的嵌入。SRAE 旨在在保留忠实度的同时重建部分原始特征空间。对 SRAE 应用了一个拉离项，使解释空间更加正交。然后引入可视化系统，用于人类理解解释空间中的特征。所提出的方法用于解释图像分类任务中的 CNN 模型，并引入了几个新的指标来定量评估解释的性能，而无需人工参与。实验表明，所提出的方法对 CNN 用于进行预测的机制产生了有趣的解释。