For spatiotemporal learning with neural networks, hyperparameters are often set manually by a human expert. This is especially the case with multiple timescale networks that require a careful setting of the values of timescales in order to learn spatiotemporal data. However, this implies a cumbersome trial-and-error process until suitable parameters are found and it reduces the long-term autonomy of artificial agents, such as robots that are controlled by multiple timescale networks. To solve the problem, we propose the evolutionary optimized multiple timescale recurrent neural network (EO-MTRNN) that is inspired by the neural plasticity of the human cortex. Our proposed network uses a method of evolutionary optimization to adjust its timescales and to rewire itself in terms of number of neurons and synapses. Moreover, it does not require additional neural networks for pre- and postprocessing input–output data. We validate our EO-MTRNN by applying it to a proposed benchmark training dataset with single and multiple sequence training cases, as well as by applying it to sensory-motor data from a robot. We compare different configuration modes of the network, and we compare the learning performance between a network configuration with manually set hyperparameters and a configuration with automatically estimated hyperparameters. The results show that automatically estimated hyperparameters yield approximately 43% better performance than manually estimated ones, without overfitting the given teaching data. We also validate the generalization ability by successfully learning data that were not included in the hyperparameter estimation process.

对于使用神经网络的时空学习，通常由人类专家手动设置超参数。对于需要仔细设置时标值以了解时空数据的多个时标网络，情况尤其如此。但是，这意味着要进行繁琐的反复试验，直到找到合适的参数为止，并且这会减少人工代理（例如由多个时标网络控制的机器人）的长期自治。为了解决该问题，我们提出了进化优化的多时标递归神经网络（EO-MTRNN）的灵感来自人类皮质的神经可塑性。我们提出的网络使用进化优化的方法来调整其时间尺度，并根据神经元和突触的数量重新布线。而且，它不需要额外的神经网络来对输入输出数据进行预处理。我们将EO-MTRNN应用于具有单个和多个序列训练案例的基准训练数据集，并将其应用于来自机器人的感觉运动数据，从而验证了EO-MTRNN。我们比较网络的不同配置模式，并比较具有手动设置的超参数的网络配置和具有自动估计的超参数的配置之间的学习性能。结果表明，在不过度拟合给定教学数据的情况下，自动估计的超参数产生的性能比手动估计的高约43％。我们还通过成功学习超参数估计过程中未包含的数据来验证泛化能力。