To continuously improve quality and reflect changes in data, machine learning applications have to regularly retrain and update their core models. We show that a differential analysis of language model snapshots before and after an update can reveal a surprising amount of detailed information about changes in the training data. We propose two new metrics---differential score and differential rank---for analyzing the leakage due to updates of natural language models. We perform leakage analysis using these metrics across models trained on several different datasets using different methods and configurations. We discuss the privacy implications of our findings, propose mitigation strategies and evaluate their effect.

中文翻译：

---

分析自然语言模型更新的信息泄漏

为了不断提高质量并反映数据的变化，机器学习应用程序必须定期重新训练和更新其核心模型。我们表明，对更新前后语言模型快照的差异分析可以揭示有关训练数据变化的大量详细信息。我们提出了两个新指标——差异分数和差异排名——用于分析由于自然语言模型更新引起的泄漏。我们使用这些指标跨使用不同方法和配置在多个不同数据集上训练的模型执行泄漏分析。我们讨论了我们的发现对隐私的影响，提出了缓解策略并评估了它们的效果。