Nonlinear differential equations rarely admit closed-form solutions, thus requiring numerical time-stepping algorithms to approximate solutions. Further, many systems characterized by multiscale physics exhibit dynamics over a vast range of timescales, making numerical integration computationally expensive due to numerical stiffness. In this work, we develop a hierarchy of deep neural network time-steppers to approximate the flow map of the dynamical system over a disparate range of time-scales. The resulting model is purely data-driven and leverages features of the multiscale dynamics, enabling numerical integration and forecasting that is both accurate and highly efficient. Moreover, similar ideas can be used to couple neural network-based models with classical numerical time-steppers. Our multiscale hierarchical time-stepping scheme provides important advantages over current time-stepping algorithms, including (i) circumventing numerical stiffness due to disparate time-scales, (ii) improved accuracy in comparison with leading neural-network architectures, (iii) efficiency in long-time simulation/forecasting due to explicit training of slow time-scale dynamics, and (iv) a flexible framework that is parallelizable and may be integrated with standard numerical time-stepping algorithms. The method is demonstrated on a wide range of nonlinear dynamical systems, including the Van der Pol oscillator, the Lorenz system, the Kuramoto-Sivashinsky equation, and fluid flow pass a cylinder; audio and video signals are also explored. On the sequence generation examples, we benchmark our algorithm against state-of-the-art methods, such as LSTM, reservoir computing, and clockwork RNN. Despite the structural simplicity of our method, it outperforms competing methods on numerical integration.

中文翻译：

---

多尺度微分方程时间步长的层次深度学习

非线性微分方程很少接受封闭形式的解，因此需要数值时间步长算法来近似解。此外，许多以多尺度物理学为特征的系统在很大的时间尺度上都表现出动力学，由于数值刚度，使得数值积分的计算量很大。在这项工作中，我们开发了一个深层神经网络时间步进器层次结构，以在不同的时标范围内近似动力学系统的流程图。生成的模型完全是数据驱动的，并利用了多尺度动力学的功能，可以进行准确而高效的数值集成和预测。此外，可以使用类似的思想将基于神经网络的模型与经典的数值时间步长相结合。与目前的时步算法相比，我们的多尺度分层时步方案具有重要的优势，包括（i）由于时标不同而规避了数值刚度；（ii）与领先的神经网络体系结构相比，准确性更高；（iii）效率高。由于对慢速时标动力学进行了显式训练，因此需要进行长时间的仿真/预测；以及（iv）可并行化并可以与标准数字时间步长算法集成的灵活框架。广泛的非线性动力学系统证明了该方法，其中包括范德波尔振子，洛伦兹系统，Kuramoto-Sivashinsky方程和通过圆柱体的流体流动。还探讨了音频和视频信号。在序列生成示例中，我们针对最新方法对我们的算法进行了基准测试，例如LSTM，储层计算和发条RNN。尽管我们的方法结构简单，但在数值积分方面却优于竞争方法。