In this research, we develop a method integrating a growing self-organizing map and differential learning system for online reinforcement learning which adaptively builds the state structure. In the conventional method, models and information on the environment are required beforehand, whereas the proposed method automatically estimates the state transitions from differentials of input signals and from these builds the state space without reference to prior information on the environment. Also, since it is an online learning method, the proposed method requires less computation and no batch memory. Through numerical experiments, we show that the proposed method has the same performance as the conventional method with the information given and that the learning time is shortened by abstraction of the state space.

在这项研究中，我们开发了一种集成增长的自组织图和差分学习系统的方法，用于在线强化学习，自适应地建立状态结构。在传统方法中，事先需要有关环境的模型和信息，而所提出的方法会自动根据输入信号的差异估计状态转换，并根据这些信号构建状态空间，而无需参考有关环境的现有信息。而且，由于它是一种在线学习方法，因此所提出的方法需要较少的计算并且不需要批处理内存。通过数值实验，我们证明了该方法在给定信息的情况下具有与传统方法相同的性能，并且通过抽象状态空间缩短了学习时间。