The vicinal risk minimization (VRM) principle is an empirical risk minimization (ERM) variant that replaces Dirac masses with vicinal functions. There is strong numerical and theoretical evidence showing that VRM outperforms ERM in terms of generalization if appropriate vicinal functions are chosen. Mixup Training (MT), a popular choice of vicinal distribution, improves generalization performance of models by introducing globally linear behavior in between training examples. Apart from generalization, recent works have shown that mixup trained models are relatively robust to input perturbations/corruptions and at same time are calibrated better than their non-mixup counterparts. In this work, we investigate the benefits of defining these vicinal distributions like mixup in latent space of generative models rather than in input space itself. We propose a new approach - - to better sample mixup images by using the latent manifold underlying the data. Our empirical studies on CIFAR-10, CIFAR-100 and Tiny-ImageNet demonstrates that models trained by performing mixup in the latent manifold learned by VAEs are inherently more robust to various input corruptions/perturbations, are significantly better calibrated and exhibit more local-linear loss landscapes.

中文翻译：

---

关于定义潜在空间中邻近分布的好处


邻近风险最小化 (VRM) 原理是一种经验风险最小化 (ERM) 变体，它用邻近函数代替狄拉克质量。有强有力的数值和理论证据表明，如果选择适当的邻位函数，VRM 在泛化方面优于 ERM。混合训练 (MT) 是邻域分布的一种流行选择，通过在训练示例之间引入全局线性行为来提高模型的泛化性能。除了泛化之外，最近的工作表明，经过混合训练的模型对于输入扰动/损坏相对稳健，同时比非混合模型的校准效果更好。在这项工作中，我们研究了定义这些邻近分布的好处，例如在生成模型的潜在空间中而不是在输入空间本身中进行混合。我们提出了一种新方法 - 通过使用数据背后的潜在流形来更好地采样混合图像。我们对 CIFAR-10、CIFAR-100 和 Tiny-ImageNet 的实证研究表明，通过在 VAE 学习到的潜在流形中执行混合来训练的模型本质上对各种输入损坏/扰动更加稳健，校准效果明显更好，并且表现出更多的局部线性损失景观。