We study the theoretical underpinning of a robust empirical risk minimization (RERM) scheme which has been finding numerous successful applications across various data science fields owing to its robustness to outliers and heavy-tailed noises. The specialties of RERM lie in its nonconvexity and that it is induced by a loss function with an integrated scale parameter trading off the robustness and the prediction accuracy. The nonconvexity of RERM and the integrated scale parameter also bring barriers when assessing its learning performance theoretically. In this paper, concerning the study of RERM, we make the following main contributions. First, we establish a no-free-lunch result, showing that there is no hope of distribution-free learning of the truth without adjusting the scale parameter. Second, by imposing the $(1+ϵ)$-th (with $ϵ>0$) order moment condition on the response variable, we establish a comparison theorem that characterizes the relation between the excess generalization error of RERM and its prediction error. Third, with a diverging scale parameter, we establish almost sure convergence rates for RERM under the $(1+ϵ)$-moment condition. Notably, the $(1+ϵ)$-moment condition allows the presence of noise with infinite variance. Last but not least, the learning theory analysis of RERM conducted in this study, on one hand, showcases the merits of RERM on robustness and the trade-off role that the scale parameter plays, and on the other hand, brings us inspirational insights into robust machine learning.

我们研究了鲁棒的经验风险最小化（RERM）方案的理论基础，该方案由于其对异常值和强尾噪声的鲁棒性而已在各种数据科学领域中找到了许多成功的应用。RERM的特长在于它的非凸性，它是由损失函数引起的，该函数具有权衡鲁棒性和预测准确性的集成比例参数。从理论上评估RERM的非凸性和积分标度参数也会带来障碍。在本文中，关于RERM的研究，我们做出了以下主要贡献。首先，我们建立了一个没有午餐的结果，表明如果不调整比例参数，就不可能免费分发真值。第二，强加$（\mathrm{1个}+ϵ）$-th（与 $ϵ>0$）在响应变量的阶矩条件下，我们建立了一个比较定理，该定理描述了RERM的过度泛化误差与其预测误差之间的关系。第三，使用不同的规模参数，我们可以确定在$（\mathrm{1个}+ϵ）$时刻条件。值得注意的是$（\mathrm{1个}+ϵ）$矩条件允许存在无限方差的噪声。最后但并非最不重要的一点是，在此研究中对RERM进行的学习理论分析，一方面展示了RERM在鲁棒性和规模参数所起的取舍作用方面的优点，另一方面，也为我们带来了启发性的见解。强大的机器学习。