当前位置: X-MOL 学术Cognit. Comput. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Efficient Implementations of Echo State Network Cross-Validation
Cognitive Computation ( IF 5.4 ) Pub Date : 2021-03-03 , DOI: 10.1007/s12559-021-09849-2
Mantas Lukoševičius , Arnas Uselis

Cross-Validation (CV) is still uncommon in time series modeling. Echo State Networks (ESNs), as a prime example of Reservoir Computing (RC) models, are known for their fast and precise one-shot learning, that often benefit from good hyper-parameter tuning. This makes them ideal to change the status quo. We discuss CV of time series for predicting a concrete time interval of interest, suggest several schemes for cross-validating ESNs and introduce an efficient algorithm for implementing them. This algorithm is presented as two levels of optimizations of doing k-fold CV. Training an RC model typically consists of two stages: (i) running the reservoir with the data and (ii) computing the optimal readouts. The first level of our optimization addresses the most computationally expensive part (i) and makes it remain constant irrespective of k. It dramatically reduces reservoir computations in any type of RC system and is enough if k is small. The second level of optimization also makes the (ii) part remain constant irrespective of large k, as long as the dimension of the output is low. We discuss when the proposed validation schemes for ESNs could be beneficial, three options for producing the final model and empirically investigate them on six different real-world datasets, as well as do empirical computation time experiments. We provide the code in an online repository. Proposed CV schemes give better and more stable test performance in all the six different real-world datasets, three task types. Empirical run times confirm our complexity analysis. In most situations, k-fold CV of ESNs and many other RC models can be done for virtually the same time and space complexity as a simple single-split validation. This enables CV to become a standard practice in RC.



中文翻译:

回声状态网络交叉验证的有效实现

交叉验证(CV)在时间序列建模中仍然不常见。回声状态网络(ESN)作为水库计算(RC)模型的主要示例,以其快速而精确的单次学习而闻名,通常受益于良好的超参数调整。这使它们成为改变现状的理想之选。我们讨论了时间序列的CV,以预测感兴趣的具体时间间隔,提出了用于交叉验证ESN的几种方案,并介绍了一种实现它们的有效算法。该算法表示为执行k的两个优化级别折简历。训练RC模型通常包括两个阶段:(i)使用数据运行储层和(ii)计算最佳读数。我们的优化的第一级解决了计算上最昂贵的部分(i),并使之与k无关地保持不变。它显着减少了任何类型的RC系统中的储层计算,如果k很小,就足够了。第二级优化也使(ii)部分保持恒定,而与k无关,只要输出的尺寸较小即可。我们讨论了提出的ESN验证方案何时可能是有益的,用于生成最终模型的三个选项,并在六个不同的现实世界数据集上进行了实证研究,以及进行了经验计算时间实验。我们在在线存储库中提供代码。建议的CV方案在所有六个不同的现实世界数据集(三种任务类型)中提供更好,更稳定的测试性能。经验运行时间证实了我们的复杂性分析。在大多数情况下,ESN和许多其他RC模型的k倍CV可以作为简单的单拆分验证在几乎相同的时间和空间复杂度下完成。这使CV成为RC中的标准做法。

更新日期:2021-03-03
down
wechat
bug