This paper addresses learning with exploration driven by chaotic internal dynamics of a neural network. Hoerzer et al. showed that a chaotic reservoir network (RN) can learn with exploration driven by external random noise and a sequential reward. In this paper, we demonstrate that a chaotic RN can learn without external noise because the output fluctuation originated from its internal chaotic dynamics functions as exploration. As learning progresses, the chaoticity decreases and the network can automatically switch from exploration mode to exploitation mode. Furthermore, the network can resume exploration when presented with a new situation. In addition, we found that even when the two parameters that influence the chaoticity are varied, learning performance always improves around the edge of chaos. From these results, we think that exploration is generated from internal chaotic dynamics, and exploitation appears in the process of forming attractors on the chaotic dynamics through learning. Consequently, exploration and exploitation are well-balanced around the edge of chaos, which leads to good learning performance.

本文探讨了由神经网络的混沌内部动力学驱动的探索学习。Hoerzer等。结果表明，混沌储层网络（RN）可以通过外部随机噪声和顺序奖励的驱动而学习。在本文中，我们证明了混沌RN可以在没有外部噪声的情况下进行学习，因为输出波动源于其内部混沌动力学的探索功能。随着学习的进行，混沌性降低，网络可以自动从探索模式切换到开发模式。此外，当出现新情况时，网络可以恢复探索。此外，我们发现，即使影响混沌的两个参数发生变化，学习性能也始终会在混沌边缘得到改善。根据这些结果，我们认为探索是由内部混沌动力学产生的，而剥削是通过学习在混沌动力学上形成吸引子的过程中出现的。因此，探索和开发在混乱的边缘是很好的平衡，这导致了良好的学习表现。