Understanding intermediate layers of a deep learning model and discovering the driving features of stimuli have attracted much interest, recently. Explainable artificial intelligence (XAI) provides a new way to open an AI black box and makes a transparent and interpretable decision. This paper proposes a new explainable convolutional neural network (XCNN) which represents important and driving visual features of stimuli in an end-to-end model architecture. This network employs encoder-decoder neural networks in a CNN architecture to represent regions of interest in an image based on its category. The proposed model is trained without localization labels and generates a heat-map as part of the network architecture without extra post-processing steps. The experimental results on the CIFAR-10, Tiny ImageNet, and MNIST datasets showed the success of our algorithm (XCNN) to make CNNs explainable. Based on visual assessment, the proposed model outperforms the current algorithms in class-specific feature representation and interpretable heatmap generation while providing a simple and flexible network architecture. The initial success of this approach warrants further study to enhance weakly supervised localization and semantic segmentation in explainable frameworks.

中文翻译：

---

卷积网络中的嵌入式编码器 - 解码器迈向可解释的 AI

最近，理解深度学习模型的中间层和发现刺激的驱动特征引起了很多兴趣。可解释人工智能 (XAI) 提供了一种打开 AI 黑匣子并做出透明和可解释决策的新方法。本文提出了一种新的可解释卷积神经网络 (XCNN)，它在端到端模型架构中代表了刺激的重要和驱动视觉特征。该网络在 CNN 架构中采用编码器-解码器神经网络，根据类别表示图像中的感兴趣区域。所提出的模型在没有定位标签的情况下进行训练，并生成热图作为网络架构的一部分，无需额外的后处理步骤。在 CIFAR-10、Tiny ImageNet、和 MNIST 数据集表明我们的算法 (XCNN) 成功地使 CNN 变得可解释。基于视觉评估，所提出的模型在特定于类的特征表示和可解释的热图生成方面优于当前算法，同时提供了简单灵活的网络架构。这种方法的初步成功值得进一步研究，以增强可解释框架中的弱监督定位和语义分割。