In machine learning, there is renewed interest in neural network ensembles (NNEs), whereby predictions are obtained as an aggregate from a diverse set of smaller models, rather than from a single larger model. Here, we show how to define and train a NNE using techniques from the study of rare trajectories in stochastic systems. We define an NNE in terms of the trajectory of the model parameters under a simple, and discrete in time, diffusive dynamics, and train the NNE by biasing these trajectories towards a small time-integrated loss, as controlled by appropriate counting fields which act as hyperparameters. We demonstrate the viability of this technique on a range of simple supervised learning tasks. We discuss potential advantages of our trajectory sampling approach compared with more conventional gradient based methods.

中文翻译：

---

通过轨迹采样训练神经网络集成

在机器学习中，人们对神经网络集成 (NNE) 重新产生了兴趣，其中预测是从一组不同的较小模型而不是从单个较大模型中获得的聚合。在这里，我们展示了如何使用随机系统中稀有轨迹研究中的技术来定义和训练 NNE。我们根据模型参数的轨迹在简单的、时间离散的、扩散动力学下定义 NNE，并通过将这些轨迹偏向小的时间积分损失来训练 NNE，由适当的计数字段控制，这些字段充当超参数。我们在一系列简单的监督学习任务中展示了这种技术的可行性。与更传统的基于梯度的方法相比，我们讨论了我们的轨迹采样方法的潜在优势。