Computation of Mutual Information (MI) helps understand the amount of information shared between a pair of random variables. Automated feature selection techniques based on MI ranking are regularly used to extract information from sensitive datasets exceeding petabytes in size, over millions of features and classes. Series of one-vs-all MI computations can be cascaded to produce n-fold MI results, rapidly pinpointing informative relationships. This ability to quickly pinpoint the most informative relationships from datasets of billions of users creates privacy concerns. In this paper, we present Distributed Differentially Private Mutual Information (DDP-MI), a privacy-safe fast batch MI, across various scenarios such as feature selection, segmentation, ranking, and query expansion. This distributed implementation is protected with global model differential privacy to provide strong assurances against a wide range of privacy attacks. We also show that our DDP-MI can substantially improve the efficiency of MI calculations compared to standard implementations on a large-scale public dataset.

中文翻译：

---

分布式差分私有互信息排序及其应用

互信息 (MI) 计算有助于了解一对随机变量之间共享的信息量。基于 MI 排名的自动特征选择技术经常用于从超过 PB 级、超过数百万个特征和类别的敏感数据集中提取信息。一系列一对多 MI 计算可以级联以产生 n 倍 MI 结果，快速确定信息关系。这种从数十亿用户的数据集中快速查明信息最丰富的关系的能力造成了隐私问题。在本文中，我们提出了分布式差分私有互信息 (DDP-MI)，这是一种隐私安全的快速批量 MI，适用于各种场景，例如特征选择、分割、排名和查询扩展。这种分布式实现受到全局模型差异隐私的保护，以提供针对广泛的隐私攻击的有力保证。我们还表明，与大规模公共数据集上的标准实现相比，我们的 DDP-MI 可以显着提高 MI 计算的效率。