With the proliferation of training data, distributed machine learning (DML) is becoming more competent for large-scale learning tasks. However, privacy concerns have to be given priority in DML, since training data may contain sensitive information of users. In this paper, we propose a privacy-preserving ADMM-based DML framework with two novel features: First, we remove the assumption commonly made in the literature that the users trust the server collecting their data. Second, the framework provides heterogeneous privacy for users depending on data's sensitive levels and servers’ trust degrees. The challenging issue is to keep the accumulation of privacy losses over ADMM iterations minimal. In the proposed framework, a local randomization approach, which is differentially private, is adopted to provide users with self-controlled privacy guarantee for the most sensitive information. Further, the ADMM algorithm is perturbed through a combined noise-adding method, which simultaneously preserves privacy for users’ less sensitive information and strengthens the privacy protection of the most sensitive information. We provide detailed analyses on the performance of the trained model according to its generalization error. Finally, we conduct extensive experiments using real-world datasets to validate the theoretical results and evaluate the classification performance of the proposed framework.

中文翻译：

---

通过局部随机化和 ADMM 扰动保护隐私的分布式机器学习

随着训练数据的激增，分布式机器学习 (DML) 越来越能胜任大规模学习任务。然而，在 DML 中必须优先考虑隐私问题，因为训练数据可能包含用户的敏感信息。在本文中，我们提出了一个基于 ADMM 的隐私保护 DML 框架，它具有两个新颖的特征：首先，我们消除了文献中常见的假设，即用户信任收集他们数据的服务器。其次，该框架根据数据的敏感级别和服务器的信任度为用户提供异构隐私。具有挑战性的问题是将 ADMM 迭代中隐私损失的累积保持在最低限度。在提议的框架中，一种局部随机化方法，它是差异私有的，为用户最敏感的信息提供自主可控的隐私保障。此外，ADMM算法通过组合噪声添加方法进行扰动，同时保护用户不太敏感的信息的隐私，并加强对最敏感信息的隐私保护。我们根据泛化误差对训练模型的性能进行详细分析。最后，我们使用真实世界的数据集进行了大量实验，以验证理论结果并评估所提出框架的分类性能。既保护了用户不太敏感的信息的隐私，又加强了对最敏感信息的隐私保护。我们根据泛化误差对训练模型的性能进行详细分析。最后，我们使用真实世界的数据集进行了大量实验，以验证理论结果并评估所提出框架的分类性能。既保护了用户不太敏感的信息的隐私，又加强了对最敏感信息的隐私保护。我们根据泛化误差对训练模型的性能进行详细分析。最后，我们使用真实世界的数据集进行了大量实验，以验证理论结果并评估所提出框架的分类性能。