The knowledge of a deep learning model may be transferred to a student model, leading to intellectual property infringement or vulnerability propagation. Detecting such knowledge reuse is nontrivial because the suspect models may not be white-box accessible and/or may serve different tasks. In this paper, we propose ModelDiff, a testing-based approach to deep learning model similarity comparison. Instead of directly comparing the weights, activations, or outputs of two models, we compare their behavioral patterns on the same set of test inputs. Specifically, the behavioral pattern of a model is represented as a decision distance vector (DDV), in which each element is the distance between the model's reactions to a pair of inputs. The knowledge similarity between two models is measured with the cosine similarity between their DDVs. To evaluate ModelDiff, we created a benchmark that contains 144 pairs of models that cover most popular model reuse methods, including transfer learning, model compression, and model stealing. Our method achieved 91.7% correctness on the benchmark, which demonstrates the effectiveness of using ModelDiff for model reuse detection. A study on mobile deep learning apps has shown the feasibility of ModelDiff on real-world models.

中文翻译：

---

ModelDiff：用于模型重用检测的基于测试的 DNN 相似性比较

深度学习模型的知识可能会转移到学生模型，导致知识产权侵权或漏洞传播。检测这种知识重用是非常重要的，因为可疑模型可能无法白盒访问和/或可能服务于不同的任务。在本文中，我们提出了 ModelDiff，这是一种基于测试的深度学习模型相似性比较方法。我们不是直接比较两个模型的权重、激活或输出，而是在同一组测试输入上比较它们的行为模式。具体来说，模型的行为模式表示为决策距离向量 (DDV)，其中每个元素是模型对一对输入的反应之间的距离。两个模型之间的知识相似度是用它们的 DDV 之间的余弦相似度来衡量的。为了评估 ModelDiff，我们创建了一个基准测试，其中包含 144 对模型，涵盖了最流行的模型重用方法，包括迁移学习、模型压缩和模型窃取。我们的方法在基准测试中实现了 91.7% 的正确率，这证明了使用 ModelDiff 进行模型重用检测的有效性。对移动深度学习应用程序的研究表明 ModelDiff 在现实世界模型上的可行性。