A long-standing goal in artificial intelligence and robotics is synthesising agents that can effectively learn and adapt throughout their lifetime. One open-ended approach to behaviour learning in autonomous robots is online evolution , which is part of the evolutionary robotics field of research. In online evolution approaches, an evolutionary algorithm is executed on the robots during task execution, which enables continuous optimisation and adaptation of behaviour. Despite the potential for automatic behaviour learning, online evolution has not been widely adopted because it often requires several hours or days to synthesise solutions to a given task. In this respect, research in the field has failed to develop a prevalent algorithm able to effectively synthesise solutions to a large number of different tasks in a timely manner. Rather than focusing on a single algorithm, we argue for more general mechanisms that can combine the benefits of different algorithms to increase the performance of online evolution of robot controllers. We conduct a comprehensive assessment of a novel approach called online hyper-evolution (OHE). Robots executing OHE use the different sources of feedback information traditionally associated with controller evaluation to find effective evolutionary algorithms during task execution. First, we study two approaches: OHE-fitness, which uses the fitness score of controllers as the criterion to select promising algorithms over time, and OHE-diversity, which relies on the behavioural diversity of controllers for algorithm selection. We then propose a novel class of techniques called OHE-hybrid, which combine diversity and fitness to search for suitable algorithms. In addition to their effectiveness at selecting suitable algorithms, the different OHE approaches are evaluated for their ability to construct algorithms by controlling which algorithmic components should be employed for controller generation (e.g., mutation, crossover, among others), an unprecedented approach in evolutionary robotics. Results show that OHE (i) facilitates the evolution of controllers with high performance, (ii) can increase effectiveness at different stages of evolution by combining the benefits of multiple algorithms over time, and (iii) can be effectively applied to construct new algorithms during task execution. Overall, our study shows that OHE is a powerful new paradigm that allows robots to improve their learning process as they operate in the task environment.

中文翻译：

---

机器人控制器在线进化的超学习算法

人工智能和机器人技术的一个长期目标是合成能够在整个生命周期内有效学习和适应的智能体。自主机器人行为学习的一种开放式方法是在线进化，这是进化机器人研究领域的一部分。在在线进化方法中，在任务执行期间在机器人上执行进化算法，从而实现行为的持续优化和适应。尽管具有自动行为学习的潜力，但在线进化并没有被广泛采用，因为它通常需要几个小时或几天来综合给定任务的解决方案。在这方面，该领域的研究未能开发出能够及时有效地综合解决大量不同任务的流行算法。与其专注于单一算法，我们主张更通用机制可以结合不同算法的优点来提高机器人控制器在线进化的性能。我们对一种称为在线超进化（哦）。执行 OHE 的机器人使用传统上与控制器评估相关的不同反馈信息源来在任务执行期间找到有效的进化算法。首先，我们研究了两种方法：OHE-fitness，它使用控制器的适应度分数作为标准来随着时间的推移选择有前途的算法，以及 OHE-diversity，它依赖于控制器的行为多样性来进行算法选择。然后，我们提出了一种称为 OHE-hybrid 的新型技术，它结合了多样性和适应度来寻找合适的算法。除了它们在选择合适算法方面的有效性外，还评估了不同的 OHE 方法的能力构造通过控制哪些算法组件应该用于控制器生成（例如，变异、交叉等）来控制算法，这是进化机器人学中前所未有的方法。结果表明，OHE (i) 促进了具有高性能的控制器的进化，(ii) 可以通过结合多种算法的优势来提高不同进化阶段的有效性，以及 (iii) 可以有效地应用于构建新的算法。任务执行。总体而言，我们的研究表明 OHE 是一种强大的新范式，它允许机器人在任务环境中运行时改进其学习过程。