We examine the stability of loss-minimizing training processes that are used for deep neural networks (DNN) and other classifiers. While a classifier is optimized during training through a so-called loss function, the performance of classifiers is usually evaluated by some measure of accuracy, such as the overall accuracy which quantifies the proportion of objects that are well classified. This leads to the guiding question of stability: does decreasing loss through training always result in increased accuracy? We formalize the notion of stability, and provide examples of instability. Our main result consists of two novel conditions on the classifier which, if either is satisfied, ensure stability of training, that is we derive tight bounds on accuracy as loss decreases. We also derive a sufficient condition for stability on the training set alone, identifying flat portions of the data manifold as potential sources of instability. The latter condition is explicitly verifiable on the training dataset. Our results do not depend on the algorithm used for training, as long as loss decreases with training.

中文翻译：

---

深度神经网络和其他分类器训练的稳定性

我们检查了用于深度神经网络 (DNN) 和其他分类器的损失最小化训练过程的稳定性。虽然分类器在训练期间通过所谓的损失函数，分类器的性能通常通过某种准确度度量来评估，例如整体准确度，它量化了分类良好的对象的比例。这导致了稳定性的指导性问题：通过训练减少损失是否总能提高准确性？我们将稳定的概念正式化，并提供不稳定的例子。我们的主要结果包括分类器上的两个新条件，如果满足其中任何一个条件，则可以确保训练的稳定性，也就是说，随着损失的减少，我们得出了准确度的严格界限。我们还得出了仅在训练集上稳定的充分条件，将数据流形的平坦部分识别为潜在的不稳定性来源。后一种条件在训练数据集上是可明确验证的。