Abstract

Convolution neural network (CNN) is a kind of powerful and efficient deep learning approach that has obtained great success in many real-world applications. However, due to its complex network structure, the intertwining of hyperparameters, and the time-consuming procedure for network training, finding an efficient network configuration for CNN is a challenging yet tough work. To efficiently solve the hyperparameters setting problem, this paper proposes a distributed particle swarm optimization (DPSO) approach, which can optimize the hyperparameters to find high-performing CNNs. Compared to tedious, historical-experience-based, and personal-preference-based manual designs, the proposed DPSO approach can evolve the hyperparameters automatically and globally to obtain promising CNNs, which provides a new idea and approach for finding the global optimal hyperparameter combination. Moreover, by cooperating with the distributed computing techniques, the DPSO approach can have a considerable speedup when compared with the traditional particle swarm optimization (PSO) algorithm. Extensive experiments on widely-used image classification benchmarks have verified that the proposed DPSO approach can effectively find the CNN model with promising performance, and at the same time, has greatly reduced the computational time when compared with traditional PSO.

摘要

卷积神经网络（CNN）是一种功能强大且高效的深度学习方法，已在许多实际应用中获得了巨大的成功。但是，由于其复杂的网络结构，超参数的交织以及网络培训的耗时过程，为CNN寻找有效的网络配置是一项艰巨而艰巨的工作。为了有效解决超参数设置问题，本文提出了一种分布式粒子群优化（DPSO）方法，可以优化超参数以找到高性能的CNN。与单调乏味的，基于历史经验的和基于个人偏好的手动设计相比，建议的DPSO方法可以自动，全局地扩展超参数以获得有希望的CNN，这为寻找全局最优超参数组合提供了新的思路和方法。此外，与传统的粒子群优化（PSO）算法相比，通过与分布式计算技术合作，DPSO方法可以大大提高速度。在广泛使用的图像分类基准上进行的大量实验证明，所提出的DPSO方法可以有效地找到性能良好的CNN模型，同时与传统的PSO相比，大大减少了计算时间。