This paper develops an efficient distributed inference algorithm, which is robust against a moderate fraction of Byzantine nodes, namely arbitrary and possibly adversarial machines in a distributed learning system. In robust statistics, the median-of-means (MOM) has been a popular approach to hedge against Byzantine failures due to its ease of implementation and computational efficiency. However, the MOM estimator has the shortcoming in terms of statistical efficiency. The first main contribution of the paper is to propose a variance reduced median-of-means (VRMOM) estimator, which improves the statistical efficiency over the vanilla MOM estimator and is computationally as efficient as the MOM. Based on the proposed VRMOM estimator, we develop a general distributed inference algorithm that is robust against Byzantine failures. Theoretically, our distributed algorithm achieves a fast convergence rate with only a constant number of rounds of communications. We also provide the asymptotic normality result for the purpose of statistical inference. To the best of our knowledge, this is the first normality result in the setting of Byzantine-robust distributed learning. The simulation results are also presented to illustrate the effectiveness of our method.

中文翻译：

---

拜占庭式鲁棒分布推断的方差减少均值中值估计

本文开发了一种高效的分布式推理算法，该算法对中等比例的拜占庭式节点（即分布式学习系统中的任意机器和可能的对抗机器）具有鲁棒性。在稳健的统计数据中，均值中值（MOM）由于易于实施和计算效率高而成为对付拜占庭式故障的一种流行方法。但是，MOM估计器在统计效率方面存在不足。本文的第一个主要贡献是提出了一种方差减少的均值中位数（VRMOM）估计器，该估计器比香草MOM估计器提高了统计效率，并且在计算上与MOM一样有效。基于拟议的VRMOM估计器，我们开发了一种通用的分布式推理算法，该算法可抵抗拜占庭式故障。从理论上讲 我们的分布式算法仅需进行恒定数量的通信即可实现快速收敛。为了统计推断，我们还提供了渐近正态性结果。据我们所知，这是拜占庭式鲁棒分布式学习的第一个常态性结果。仿真结果也被提出来说明我们方法的有效性。