In recent years, research in applying optimization approaches in the automatic design of deep neural networks has become increasingly popular. Although various approaches have been proposed, there is a lack of a comprehensive survey and taxonomy on this hot research topic. In this article, we propose a unified way to describe the various optimization algorithms that focus on common and important search components of optimization algorithms: representation, objective function, constraints, initial solution(s), and variation operators. In addition to large-scale search space, the problem is characterized by its variable mixed design space, it is very expensive, and it has multiple blackbox objective functions. Hence, this unified methodology has been extended to advanced optimization approaches, such as surrogate-based, multi-objective, and parallel optimization.

中文翻译：

---

深度神经网络的自动化设计

近年来，在深度神经网络的自动设计中应用优化方法的研究越来越受欢迎。尽管已经提出了各种方法，但缺乏对这一热门研究课题的全面调查和分类。在本文中，我们提出了一种统一的方法来描述各种优化算法，这些优化算法专注于优化算法的常见和重要搜索组件：表示、目标函数、约束、初始解决方案和变分算子。除了大规模搜索空间之外，该问题的特点是其可变混合设计空间，非常昂贵，并且具有多个黑盒目标函数。因此，这种统一的方法已扩展到高级优化方法，例如基于代理的、多目标的、