The classical problem of supervised learning is to infer an accurate estimate of a target variable $Y$ from a measured variable $X$ using a set of labeled training samples. Motivated by the increasingly distributed nature of data and decision making, this paper considers a variation of this classical problem in which the inference is distributed between two nodes, e.g., a mobile device and a cloud, with a rate constraint on the communication between them. The mobile device observes $X$ and sends a description $M$ of $X$ to the cloud, which computes an estimate $\hat {Y}$ of $Y$ . We follow the recent minimax learning approach to study this inference problem and show that it corresponds to a one-shot minimax noisy lossy source coding problem. We then establish information theoretic bounds on the risk-rate Lagrangian cost, leading to a general method for designing a near-optimal descriptor-estimator pair. A key ingredient in the proof of our result is a refined version of the strong functional representation lemma previously used to establish several one-shot source coding theorems. Our results show that a naive estimate-compress scheme for rate-constrained inference is not optimal in general. When the distribution of $(X,Y)$ is known and the error is measured by the logarithmic loss, our bounds on the risk-rate Lagrangian cost provide a new one-shot operational interpretation of the information bottleneck. We also demonstrate a way to bound the excess risk of the descriptor-estimator pair obtained by our method.

中文翻译：

---

分布式推理的极小极大学习

监督学习的经典问题是推断目标变量的准确估计 $Y$ 从测量变量 $X$ 使用一组标记的训练样本。受数据和决策越来越分散的性质的启发，本文考虑了这个经典问题的变体，其中推理分布在两个节点之间，例如移动设备和云，对它们之间的通信有速率限制。移动设备观察 $X$ 并发送说明 百万美元 的 $X$ 到云，计算估计 $\hat {Y}$ 的 $Y$ . 我们遵循最近的极小极大学习方法来研究这个推理问题，并表明它对应于单次极小极大噪声有损源编码问题。然后，我们建立了风险率拉格朗日成本的信息理论界限，从而得出了一种设计近乎最优的描述符-估计量对的通用方法。证明我们结果的一个关键因素是先前用于建立几个一次性源编码定理的强函数表示引理的改进版本。我们的结果表明，用于速率约束推理的朴素估计压缩方案通常不是最佳的。当分布 $(X,Y)$ 已知并且误差是通过对数损失来衡量的，我们对风险率拉格朗日成本的界限提供了对信息瓶颈的新的一次性操作解释。我们还演示了一种限制通过我们的方法获得的描述符-估计器对的过度风险的方法。