The last decade has witnessed a rapid advance in machine learning models. While the black-box nature of these systems allows powerful predictions, it cannot be directly explained, posing a threat to the continuing democratization of machine learning technology. Tackling the challenge of model explainability, research has made significant progress in demystifying the image classification models. In the same spirit of these works, this paper studies code summarization models, particularly, given an input program for which a model makes a prediction, our goal is to reveal the key features that the model uses for predicting the label of the program. We realize our approach in HouYi, which we use to evaluate four prominent code summarization models: extreme summarizer, code2vec, code2seq, and sequence GNN. Results show that all models base their predictions on syntactic and lexical properties with little to none semantic implication. Based on this finding, we present a novel approach to explaining the predictions of code summarization models through the lens of training data. Our work opens up this exciting, new direction of studying what models have learned from source code.

中文翻译：

---

揭秘代码摘要模型

过去十年见证了机器学习模型的飞速发展。尽管这些系统的黑盒性质允许进行有力的预测，但无法直接进行解释，这对机器学习技术的持续民主化构成了威胁。为应对模型可解释性的挑战，研究在使图像分类模型神秘化方面取得了重大进展。本着这些工作的精神，本文研究代码摘要模型，特别是在给定模型可对其进行预测的输入程序的情况下，我们的目标是揭示模型用于预测程序标签的关键特征。我们在后一实现了我们的方法，该方法用于评估四个杰出的代码汇总模型：极限汇总器，code2vec，code2seq和序列GNN。结果表明，所有模型的预测都是基于句法和词汇特性的，几乎没有语义暗示。基于此发现，我们提出了一种新颖的方法，可以通过训练数据的角度来解释代码摘要模型的预测。我们的工作为研究模型从源代码中学到了新的方向开辟了令人兴奋的方向。