Sharing a pre-trained machine learning model, particularly a deep neural network via prediction APIs, is becoming a common practice on machine learning as a service (MLaaS) platforms nowadays. Although deep neural networks (DNN) have shown remarkable successes in many tasks, they are also criticized for the lack of interpretability and transparency. Interpreting a shared DNN model faces two additional challenges compared with interpreting a general model. (1) Limited training data can be disclosed to users. (2) The internal structure of the models may not be available. These two challenges impede the application of most existing interpretability approaches, such as saliency maps or influence functions, for DNN models. Case-based reasoning methods have been used for interpreting decisions; however, how to select and organize the data points under the constraints of shared DNN models is not discussed. Moreover, simply providing cases as explanations may not be sufficient for supporting instance level interpretability. Meanwhile, existing interpretation methods for DNN models generally lack the means to evaluate the reliability of the interpretation. In this article, we propose a framework named Shared Model INTerpreter (SMINT) to address the above limitations. We propose a new data structure called a boundary graph to organize training points to mimic the predictions of DNN models. We integrate local features, such as saliency maps and interpretable input masks, into the data structure to help users to infer the model decision boundaries. We show that the boundary graph is able to address the reliability issues in many local interpretation methods. We further design an algorithm named hidden-layer aware p-test to measure the reliability of the interpretations. Our experiments show that SMINT is able to achieve above 99% fidelity to corresponding DNN models on both MNIST and ImageNet by sharing only a tiny fraction of training data to make these models interpretable. The human pilot study demonstrates that SMINT provides better interpretability compared with existing methods. Moreover, we demonstrate that SMINT is able to assist model tuning for better performance on different user data.

中文翻译：

---

精明

共享预先训练的机器学习模型，特别是通过预测 API 共享深度神经网络，正在成为当今机器学习即服务 (MLaaS) 平台的常见做法。尽管深度神经网络 (DNN) 在许多任务中都取得了显著成功，但它们也因缺乏可解释性和透明度而受到批评。与解释通用模型相比，解释共享 DNN 模型面临两个额外的挑战。(1) 可以向用户披露有限的训练数据。(2) 模型的内部结构可能不可用。这两个挑战阻碍了大多数现有可解释性方法（例如显着性图或影响函数）在 DNN 模型中的应用。基于案例的推理方法已被用于解释决策；然而，没有讨论如何在共享 DNN 模型的约束下选择和组织数据点。此外，仅提供案例作为解释可能不足以支持实例级别的可解释性。同时，现有的 DNN 模型解释方法普遍缺乏评估解释可靠性的手段。在本文中，我们提出了一个名为 Shared Model INTerpreter (SMINT) 的框架来解决上述限制。我们提出了一种称为边界图的新数据结构来组织训练点以模仿 DNN 模型的预测。我们将局部特征（例如显着性图和可解释的输入掩码）集成到数据结构中，以帮助用户推断模型决策边界。我们表明边界图能够解决许多局部解释方法中的可靠性问题。我们进一步设计了一种名为隐藏层感知 p 检验的算法来测量解释的可靠性。我们的实验表明，通过仅共享一小部分训练数据以使这些模型可解释，SMINT 能够在 MNIST 和 ImageNet 上实现对相应 DNN 模型的 99% 以上的保真度。人类试点研究表明，与现有方法相比，SMINT 提供了更好的可解释性。此外，我们证明了 SMINT 能够协助模型调整，以便在不同的用户数据上获得更好的性能。我们的实验表明，通过仅共享一小部分训练数据以使这些模型可解释，SMINT 能够在 MNIST 和 ImageNet 上实现对相应 DNN 模型的 99% 以上的保真度。人类试点研究表明，与现有方法相比，SMINT 提供了更好的可解释性。此外，我们证明了 SMINT 能够协助模型调整，以便在不同的用户数据上获得更好的性能。我们的实验表明，通过仅共享一小部分训练数据以使这些模型可解释，SMINT 能够在 MNIST 和 ImageNet 上实现对相应 DNN 模型的 99% 以上的保真度。人类试点研究表明，与现有方法相比，SMINT 提供了更好的可解释性。此外，我们证明了 SMINT 能够协助模型调整，以便在不同的用户数据上获得更好的性能。