Echo State Networks (ESNs) are a class of single-layer recurrent neural networks with randomly generated internal weights, and a single layer of tuneable outer weights, which are usually trained by regularised linear least squares regression. Remarkably, ESNs still enjoy the universal approximation property despite the training procedure being entirely linear. In this paper, we prove that an ESN trained on a sequence of observations from an ergodic dynamical system (with invariant measure $\mu$) using Tikhonov least squares regression against a set of targets, will approximate the target function in the $L^2\left(\mu \right)$ norm. In the special case that the targets are future observations, the ESN is learning the next step map, which allows time series forecasting. We demonstrate the theory numerically by training an ESN using Tikhonov least squares on a sequence of scalar observations of the Lorenz system.

回声状态网络（ESN）是一类具有随机生成的内部权重和单层可调整外部权重的单层递归神经网络，通常通过正则化线性最小二乘回归训练它们。值得注意的是，尽管训练过程完全是线性的，但ESN仍具有通用逼近性。在本文中，我们证明了ESN是根据遍历动力系统（具有不变测度）的一系列观测值进行训练的$\mu$）对一组目标使用Tikhonov最小二乘回归，将在 ${\mathrm{大号}}^{\mathrm{2个}}（\mu ）$规范。在特殊情况下，目标是将来的观测，ESN正在学习下一步地图，该地图可以进行时间序列预测。我们通过在Lorenz系统的标量观测序列上使用Tikhonov最小二乘训练ESN来通过数值方法论证该理论。