Abstract. Neural networks are able to approximate chaotic dynamical systems when provided with training data that cover all relevant regions of the system's phase space. However, many practical applications diverge from this idealized scenario. Here, we investigate the ability of feed-forward neural networks to (1) learn the behavior of dynamical systems from incomplete training data and (2) learn the influence of an external forcing on the dynamics. Climate science is a real-world example where these questions may be relevant: it is concerned with a non-stationary chaotic system subject to external forcing and whose behavior is known only through comparatively short data series. Our analysis is performed on the Lorenz63 and Lorenz95 models. We show that for the Lorenz63 system, neural networks trained on data covering only part of the system's phase space struggle to make skillful short-term forecasts in the regions excluded from the training. Additionally, when making long series of consecutive forecasts, the networks struggle to reproduce trajectories exploring regions beyond those seen in the training data, except for cases where only small parts are left out during training. We find this is due to the neural network learning a localized mapping for each region of phase space in the training data rather than a global mapping. This manifests itself in that parts of the networks learn only particular parts of the phase space. In contrast, for the Lorenz95 system the networks succeed in generalizing to new parts of the phase space not seen in the training data. We also find that the networks are able to learn the influence of an external forcing, but only when given relatively large ranges of the forcing in the training. These results point to potential limitations of feed-forward neural networks in generalizing a system's behavior given limited initial information. Much attention must therefore be given to designing appropriate train-test splits for real-world applications.

中文翻译：

---

在洛伦兹系统上训练的前馈神经网络的泛化特性

摘要。当提供覆盖系统相空间所有相关区域的训练数据时，神经网络能够逼近混沌动力系统。然而，许多实际应用与这种理想化场景不同。在这里，我们研究了前馈神经网络的能力，以 (1) 从不完整的训练数据中学习动力系统的行为，以及 (2) 学习外力对动力学的影响。气候科学是一个现实世界的例子，这些问题可能与这些问题相关：它涉及一个受外力影响的非平稳混沌系统，其行为只能通过相对较短的数据系列才能知道。我们的分析是在 Lorenz63 和 Lorenz95 模型上进行的。我们证明，对于 Lorenz63 系统，对仅覆盖系统相空间一部分的数据进行训练的神经网络难以对排除在训练之外的区域进行熟练的短期预测。此外，在进行长系列连续预测时，网络难以重现探索训练数据中所见区域之外的轨迹，除非在训练期间只遗漏了小部分的情况。我们发现这是由于神经网络为训练数据中的每个相空间区域学习局部映射而不是全局映射。这表现在部分网络只学习相空间的特定部分。相比之下，对于 Lorenz95 系统，网络成功地推广到训练数据中看不到的相空间的新部分。我们还发现网络能够学习外部强迫的影响，但前提是在训练中给予相对较大的强迫范围。这些结果表明前馈神经网络在给定有限的初始信息的情况下概括系统行为的潜在局限性。因此，必须非常注意为实际应用设计合适的训练测试分割。