In deep learning, deep neural network (DNN) hyperparameters can severely affect network performance. Currently, such hyperparameters are frequently optimized by several methods, such as Bayesian optimization and the covariance matrix adaptation evolution strategy. However, it is difficult for non-experts to employ these methods. In this paper, we adapted the simpler coordinate-search and Nelder-Mead methods to optimize hyperparameters. Several hyperparameter optimization methods were compared by configuring DNNs for character recognition and age/gender classification. Numerical results demonstrated that the Nelder-Mead method outperforms the other methods and achieves state-of-the-art accuracy for age/gender classification.

中文翻译：

---

在深度学习中使用Nelder-Mead方法进行有效的超参数优化

在深度学习中，深度神经网络（DNN）超参数会严重影响网络性能。当前，此类超参数经常通过几种方法来优化，例如贝叶斯优化和协方差矩阵适应进化策略。但是，非专业人员很难采用这些方法。在本文中，我们采用了更简单的坐标搜索和Nelder-Mead方法来优化超参数。通过配置DNN进行字符识别和年龄/性别分类，比较了几种超参数优化方法。数值结果表明，Nelder-Mead方法优于其他方法，并且在年龄/性别分类方面达到了最新的准确性。