In this paper, traditional and meta-heuristic approaches for optimizing deep neural networks (DNN) have been surveyed, and a genetic algorithm (GA)-based approach involving two optimization phases for hyper-parameter discovery and optimal data subset determination has been proposed. The first phase aims to quickly select an optimal combination of the network hyper-parameters to design a DNN. Compared to the traditional grid-search-based method, the optimal parameters have been computed 6.5 times faster for recurrent neural network (RNN) and 8 times faster for convolutional neural network (CNN). The proposed approach is capable of tuning multiple hyper-parameters simultaneously. The second phase finds an appropriate subset of the training data for near-optimal prediction performance, providing an additional speedup of 75.86% for RNN and 41.12% for CNN over the first phase.

本文研究了用于优化深度神经网络（DNN）的传统方法和元启发式方法，并提出了一种基于遗传算法（GA）的方法，该方法涉及两个用于优化参数发现和最佳数据子集确定的优化阶段。第一阶段旨在快速选择网络超参数的最佳组合以设计DNN。与传统的基于网格搜索的方法相比，递归神经网络（RNN）的最优参数计算速度快了6.5倍，卷积神经网络（CNN）的最优参数计算速度快了8倍。所提出的方法能够同时调整多个超参数。第二阶段找到训练数据的合适子集以实现接近最佳的预测性能，从而为RNN和41提供75.86％的额外加速。