Although deep learning algorithms have been widely used, their performance depends heavily on a good set of hyperparameters. This article presents an efficient Hyperband and trust-region-based mode-pursuing sampling hybrid method for hyperparameter optimization. First, Hyperband is modified and used to select the optimum quickly from a large number of random sampling points to construct a trust region. Secondly, mode-pursuing sampling is performed in the trust region to generate more points systematically around the minimum, and the location or size of the trust region is dynamically adjusted to accelerate its convergence. Thirdly, the process of selection and sampling is repeated until a termination criterion is met. Numerical examples are presented to verify the effectiveness of the hybrid method, the results of which are compared with those of five well-known algorithms. Comparison results show that better optimal solutions are obtained through the hybrid method, with a higher efficiency.

尽管深度学习算法已被广泛使用，但它们的性能在很大程度上取决于一组良好的超参数。本文提出了一种用于超参数优化的有效的基于超频带和信任区域的模式追求采样混合方法。首先，修改超频带并用于从大量随机采样点中快速选择最佳值，以构建信任区域。其次，在信任区域中进行模式追求采样以在最小值附近系统地生成更多点，并且动态调整信任区域的位置或大小以加速其收敛。第三，重复选择和采样的过程，直到满足终止标准为止。数值例子验证了混合方法的有效性，将其结果与五种著名算法的结果进行比较。比较结果表明，通过混合法可以获得更好的最优解，效率更高。