Abstract
The Conceptor network is a new framework of reservoir computing (RC), in addition to the features of easy training, global convergence, it can online learn new classes of input patterns without complete re-learning from all the training data. The conventional connection topology and weights of the hidden layer (reservoir) of RC are initialized randomly, and are fixed to be no longer fine-tuned after initialization. However, it has been demonstrated that the reservoir connection of RC plays an important role in the computational performance of RC. Therefore, in this paper, we optimize the Conceptor’s reservoir connection and propose a phase space reconstruction (PSR) -based reservoir generation method. We tested the generation method on time series prediction task, and the experiment results showed that the proposed PSR-based method can improve the prediction accuracy of Conceptor networks. Further, we compared the PSR-based Conceptor with two Conceptor networks of other typical reservoir topologies (random connected, cortex-like connected), and found that all of their prediction accuracy showed a nonlinear decline trend with increasing storage load, but in comparison, our proposed PSR-based method has the best accuracy under different storage loads.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Chen Q, Zhang A, Huang T, He Q, Song Y (2018) Imbalanced dataset-based echo state networks for anomaly detection. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3747-z
Deng Z, Zhang Y (2007) Collective behavior of a small-world recurrent neural system with scale-free distribution. In: IEEE Transactions on neural networks, pp 1364–1375
Ding H, Pei W, He Z (2005) A multiple objective optimization based echo state network tree and application to intrusion detection. IEEE Int Workshop VLSl Des Video Technol 52:443–446
Gao Z, Jin N (2009) Complex network from time series based on phase space reconstruction. Chaos 19(3):033137. https://doi.org/10.1063/1.3227736
Hu H, Wang L, Lv SX (2020) Forecasting energy consumption and wind power generation using deep echo state network. Renew Energy 154:598–613. https://doi.org/10.1016/j.renene.2020.03.042
Hu Y, Ishwarya M, Kiong LC (2015) Classify images with conceptor network. cs.CV, arXiv:1506.00815
Jaeger H (2014) Controlling recurrent neural networks by conceptors. Technical Report
Jaeger H, Haas H (2004) Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304:78–80. https://doi.org/10.1126/science.1091277
Jaeger H, Lukosevieius M, Popovici D, Sieweret U (2007) Optimization and applications of echo state networks with leaky integrator neurons. Neural Netw 20(3):335–352. https://doi.org/10.1016/j.neunet.2007.04.016
Li X, Chen Q, Xue F (2016) Bursting dynamics remarkably improve the performance of neural networks on liquid computing. Cogn Neurodyn 10:415–421. https://doi.org/10.1007/s11571-016-9387-z
Li X, Zhong L, Xue F, Zhang A (2015) A priori data-driven multi-clustered reservoir generation algorithm for echo state network. PLoS ONE 10(4):e0120750. https://doi.org/10.1371/journal.pone.0120750
Liu T, Sedoc J, Ungar L (2018) Correcting the common discourse bias in linear representation of sentences using conceptors. Proc BioCreative/OHNLP Challenge 2018:250–256
M E, L A, J, L (2009) Reservoir computing for static pattern recognition. In: European symposium on artificial neural network, pp 245–250. https://doi.org/10.1002/0470848944.hsa115
Ma Q, Chen W (2013) Modular state space of echo state network. Neurocomputing 122:406–417
Ma Q, Zheng Q, Peng H, Qin J (2009) Chaotic time series prediction based on fuzzy boundary modular neural networks. Acta Physics Sinca 58(3):1410. https://doi.org/10.1109/ICMLC.2007.4370752
Maass W, Natschlager T, Markram H (2002) Real-time computing without stable states: a new framework for neural computation based on perturbatiorns. Neural Comput 14(11):2531–2560. https://doi.org/10.1162/089976602760407955
Malagarriga D, Pons AJ, Villa AEP (2019) Complex temporal patterns processing by a neural mass model of a cortical column. Cogn Neurodyn 13:379–392. https://doi.org/10.1007/s11571-019-09531-2
Mohammadpoory Z, Nasrolahzadeh M, Mahmoodian N, Sayyah M, Haddadnia J (2019) Complex network based models of ecog signals for detectionof induced epileptic seizures in rats. Cog Neurodyn. https://doi.org/10.1007/s11571-019-09527-y
Qian G, Zhang L (2018) A simple feedforward convolutional conceptor neural network for classification. Appl Soft Comput 70:1034–1041. https://doi.org/10.1016/j.asoc.2017.08.016
Qian G, Zhang L, Zhang Q (2018) End-to-end training algorithm for conceptor-based neural networks. Electron Lett 54(15):924–926. https://doi.org/10.1049/el.2018.0033
Skowronski MD, Harris JG (2007) Automatic speech recognition using a predictive echo state network classifier. Neural Netw 20:414–423. https://doi.org/10.1016/j.neunet.2007.04.006
Wang L, Wang Z, Liu S (2016) An effective multivariate time series classification approach using echo state network and adaptive differential evolution algorithm. Expert Syst Appl 43:237–249. https://doi.org/10.1016/j.eswa.2015.08.055
Wang Z, Zeng YR, Wang S, Wang L (2019) Optimizing echo state network with backtracking search optimization algorithm for time series forecasting. Eng Appl Artif Intell 81:117–132. https://doi.org/10.1016/j.engappai.2019.02.009
Xu Z, Zhong L, Zhang A (2019) Phase space reconstruction-based conceptor network for time series prediction. IEEE Access 7:163172–163179. https://doi.org/10.1109/ACCESS.2019.2952365
Yang Y, Yang H (2008) Complex network-based time series analysis. Phys A 387:1381–1386. https://doi.org/10.1016/j.physa.2007.10.055
Zhang A, Zhu W, Liu M (2017) Self-organizing reservoir computing based on spiking-timing dependent plasticity and intrinsic plasticity mechanisms. In: Chinese automation congress (CAC), vol 2017, pp 6189–6193. IEEE. https://doi.org/10.1109/CAC.2017.8243892. http://ieeexplore.ieee.org/document/8243892/
Zhang A, Zhu W, Li J (2019) Spiking echo state convolutional neural network for robust time series classification. IEEE Access 7:4927–4935. https://doi.org/10.1109/ACCESS.2018.2887354
Zhang J, Small M (2006) Complex network from pseudoperiodic time series: topology versus dynamics. Phys Rev Lett 96:238701. https://doi.org/10.1103/PhysRevLett.96.238701
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, A., Xu, Z. Chaotic time series prediction using phase space reconstruction based conceptor network. Cogn Neurodyn 14, 849–857 (2020). https://doi.org/10.1007/s11571-020-09612-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11571-020-09612-7