Deep neural networks (DNNs), which are extensions of artificial neural networks, can learn higher levels of feature hierarchy established by lower level features by transforming the raw feature space to another complex feature space. Although deep networks are successful in a wide range of problems in different fields, there are some issues affecting their overall performance such as selecting appropriate values for model parameters, deciding the optimal architecture and feature representation and determining optimal weight and bias values. Recently, metaheuristic algorithms have been proposed to automate these tasks. This survey gives brief information about common basic DNN architectures including convolutional neural networks, unsupervised pre-trained models, recurrent neural networks and recursive neural networks. We formulate the optimization problems in DNN design such as architecture optimization, hyper-parameter optimization, training and feature representation level optimization. The encoding schemes used in metaheuristics to represent the network architectures are categorized. The evolutionary and selection operators, and also speed-up methods are summarized, and the main approaches to validate the results of networks designed by metaheuristics are provided. Moreover, we group the studies on the metaheuristics for deep neural networks based on the problem type considered and present the datasets mostly used in the studies for the readers. We discuss about the pros and cons of utilizing metaheuristics in deep learning field and give some future directions for connecting the metaheuristics and deep learning. To the best of our knowledge, this is the most comprehensive survey about metaheuristics used in deep learning field.

深度神经网络（DNN）是人工神经网络的扩展，可以通过将原始特征空间转换为另一个复杂的特征空间来学习由较低层特征建立的较高层特征层次。尽管深层网络在不同领域中的许多问题上都取得了成功，但仍有一些问题会影响其整体性能，例如为模型参数选择适当的值，确定最佳的体系结构和特征表示以及确定最佳的权重和偏差值。最近，已经提出了元启发式算法来自动化这些任务。这项调查简要介绍了常见的基本DNN架构，包括卷积神经网络，无监督的预训练模型，递归神经网络和递归神经网络。我们提出了DNN设计中的优化问题，例如架构优化，超参数优化，训练和特征表示级别优化。在元启发法中用来表示网络体系结构的编码方案已分类。总结了进化算子和选择算子，以及提速方法，并提供了验证元启发式设计网络结果的主要方法。此外，我们根据所考虑的问题类型对深层神经网络的元启发式方法进行了分组研究，并为读者提供了研究中最常用的数据集。我们讨论了在深度学习领域中使用元启发式方法的利弊，并给出了将元启发式方法与深度学习联系起来的一些未来方向。据我们所知，