Echo State Networks (ESNs) are a class of single-layer recurrent neural networks that have enjoyed recent attention. In this paper we prove that a suitable ESN, trained on a series of measurements of an invertible dynamical system, induces a C1 map from the dynamical system's phase space to the ESN's reservoir space. We call this the Echo State Map. We then prove that the Echo State Map is generically an embedding with positive probability. Under additional mild assumptions, we further conjecture that the Echo State Map is almost surely an embedding. For sufficiently large, and specially structured, but still randomly generated ESNs, we prove that there exists a linear readout layer that allows the ESN to predict the next observation of a dynamical system arbitrarily well. Consequently, if the dynamical system under observation is structurally stable then the trained ESN will exhibit dynamics that are topologically conjugate to the future behaviour of the observed dynamical system. Our theoretical results connect the theory of ESNs to the delay-embedding literature for dynamical systems, and are supported by numerical evidence from simulations of the traditional Lorenz equations. The simulations confirm that, from a one dimensional observation function, an ESN can accurately infer a range of geometric and topological features of the dynamics such as the eigenvalues of equilibrium points, Lyapunov exponents and homology groups.

中文翻译：

---

回波状态网络的嵌入和逼近定理。

回声状态网络（ESN）是一类最近受到关注的单层递归神经网络。在本文中，我们证明了经过对可逆动力学系统的一系列测量进行训练的合适ESN，可以从动力学系统的相空间到ESN的储层空间生成C1映射。我们称此为“回声州地图”。然后，我们证明回声状态图通常是具有正概率的嵌入。在其他温和的假设下，我们进一步推测“回声状态图”几乎可以肯定是嵌入的。对于足够大且经过特殊构造但仍随机生成的ESN，我们证明存在一个线性读出层，该层可使ESN任意预测动态系统的下一个观测值。所以，如果所观察的动力学系统在结构上是稳定的，则受过训练的ESN将显示与所观察到的动力学系统的未来行为在拓扑上共轭的动力学。我们的理论结果将ESN的理论与动态系统的延迟嵌入文献联系起来，并得到了传统Lorenz方程仿真的数值证据的支持。仿真证实，从一维观测函数来看，ESN可以准确地推断动力学的一系列几何和拓扑特征，例如平衡点的特征值，Lyapunov指数和同源性组。我们的理论结果将ESN的理论与动力系统的延迟嵌入文献联系起来，并得到了传统Lorenz方程仿真的数值证据的支持。仿真证实，从一维观测函数来看，ESN可以准确地推断动力学的一系列几何和拓扑特征，例如平衡点的特征值，Lyapunov指数和同源性组。我们的理论结果将ESN的理论与动态系统的延迟嵌入文献联系起来，并得到了传统Lorenz方程仿真的数值证据的支持。仿真证实，从一维观测函数来看，ESN可以准确地推断动力学的一系列几何和拓扑特征，例如平衡点的特征值，Lyapunov指数和同源性组。