The phenomenon of benign overfitting is one of the key mysteries uncovered by deep learning methodology: deep neural networks seem to predict well, even with a perfect fit to noisy training data. Motivated by this phenomenon, we consider when a perfect fit to training data in linear regression is compatible with accurate prediction. We give a characterization of linear regression problems for which the minimum norm interpolating prediction rule has near-optimal prediction accuracy. The characterization is in terms of two notions of the effective rank of the data covariance. It shows that overparameterization is essential for benign overfitting in this setting: the number of directions in parameter space that are unimportant for prediction must significantly exceed the sample size. By studying examples of data covariance properties that this characterization shows are required for benign overfitting, we find an important role for finite-dimensional data: the accuracy of the minimum norm interpolating prediction rule approaches the best possible accuracy for a much narrower range of properties of the data distribution when the data lie in an infinite-dimensional space vs. when the data lie in a finite-dimensional space with dimension that grows faster than the sample size.

良性过度拟合现象是深度学习方法揭开的关键谜团之一：深度神经网络似乎可以很好地预测，即使完美地拟合了噪声训练数据。受这种现象的启发，我们考虑线性回归中训练数据的完美拟合何时与准确预测兼容。我们给出了线性回归问题的特征，其中最小范数插值预测规则具有接近最优的预测精度。表征是根据数据协方差的有效等级的两个概念。它表明，在此设置中，过度参数化对于良性过度拟合至关重要：参数空间中对预测不重要的方向数量必须显着超过样本大小。通过研究这种表征显示的良性过度拟合所需的数据协方差属性的示例，我们发现有限维数据的重要作用：最小范数插值预测规则的准确性接近于更窄范围的属性的最佳可能准确性。数据位于无限维空间时的数据分布与数据位于维度增长快于样本大小的有限维空间时的数据分布。