Evolutionary optimization aims to tune the hyper-parameters during learning in a computationally fast manner. For optimization of multi-task problems, evolution is done by creating a unified search space with a dimensionality that can include all the tasks. Multi-task evolution is achieved via selective imitation where two individuals with the same type of skill are encouraged to crossover. Due to the relatedness of the tasks, the resulting offspring may have a skill for a different task. In this way, we can simultaneously evolve a population where different individuals excel in different tasks. In this paper, we consider a type of evolution called Genetic Programming (GP) where the population of genes have a tree-like structure and can be of different lengths and hence can naturally represent multiple tasks. We apply the model to multi-task neuroevolution that aims to determine the optimal hyper-parameters of a neural network such as number of nodes, learning rate, and number of training epochs using evolution. Here each gene is encoded with the hyper parameters for a single neural network. Previously, optimization was done by enabling or disabling individual connections between neurons during evolution. This method is extremely slow and does not generalize well to new neural architectures such as Seq2Seq. To overcome this limitation, we follow a modular approach where each sub-tree in a GP can be a sub-neural architecture that is preserved during crossover across multiple tasks. Lastly, in order to leverage on the inter-task covariance for faster evolutionary search, we project the features from both tasks to common space using fuzzy membership functions. The proposed model is used to determine the optimal topology of a feed-forward neural network for classification of emotions in physiological heart signals and also a Seq2seq chatbot that can converse with kindergarten children. We can outperform baselines by over 10% in accuracy.

进化优化旨在以计算快速的方式在学习过程中调整超参数。为了优化多任务问题，通过创建一个统一的搜索空间来进行进化，该搜索空间的维数可以包含所有任务。多任务进化是通过选择性模仿来实现的，其中鼓励具有相同技能类型的两个人进行交叉。由于任务的相关性，因此产生的后代可能具有执行不同任务的技能。通过这种方式，我们可以同时演化出一个群体，其中不同的个人在不同的任务中表现出色。在本文中，我们考虑一种称为遗传编程（GP）的进化类型，其中基因种群具有树状结构，并且可以具有不同的长度，因此自然可以代表多个任务。我们将模型应用于多任务神经进化，旨在确定神经网络的最佳超参数，例如节点数，学习率和使用进化的训练时期数。在这里，每个基因都用单个神经网络的超参数编码。以前，优化是通过在进化过程中启用或禁用神经元之间的各个连接来完成的。该方法非常慢，并且不能很好地推广到新的神经体系结构，例如Seq2Seq。为了克服此限制，我们采用模块化方法，其中GP中的每个子树可以是在跨多个任务的交叉过程中保留的子神经体系结构。最后，为了利用任务间的协方差实现更快的进化搜索，我们使用模糊隶属函数将特征从这两个任务投影到公共空间。所提出的模型用于确定用于对生理性心脏信号中的情绪进行分类的前馈神经网络的最佳拓扑，以及可以与幼儿园儿童交谈的Seq2seq聊天机器人。我们可以超过基准10倍以上精度百分比。