The efficacy and simplicity of using only function evaluations in zeroth-order stochastic optimization (ZOSO) makes it achieve great attention in solving large scale learning tasks. However, the question of how to choose an appropriate step size sequence timely for ZOSO has been less researched. To fill this defect, this paper provides a fast automatic step size selection approach by using an improved Barzilai-Borwein (IBB) technique with the zeroth-order (ZO) gradient estimator for ZOSO. In detail, this work shows the efficacy of the IBB technique by applying it into the advanced ZOSO method, i.e., zeroth-order stochastic variance reduced gradient (ZO-SVRG) method, which leads to a method: ZO-SVRG-IBB. We theoretically analyze the convergence of the ZO-SVRG-IBB method under the random gradient estimator and the coordinate-wise gradient estimator settings, respectively, and show that the convergence rate of ZO-SVRG-IBB matches the best-known convergence rate of advanced ZOSO methods on nonconvex functions. We further show that the query complexity of ZO-SVRG-IBB is comparable to advanced ZOSO methods. Extensive numerical experiments performed on different datasets show that the proposed method outperforms or matches state-of-the-art ZOSO methods.

在零阶随机优化（ZOSO）中仅使用函数评估的功效和简便性使其在解决大规模学习任务时引起了极大的关注。但是，关于如何为ZOSO及时选择合适的步长序列的问题研究较少。为了弥补这一缺陷，本文提供了一种通过使用改进的Barzilai-Borwein（IBB）技术和ZOSO的零阶（ZO）梯度估计器提供的快速自动步长选择方法。详细地讲，这项工作通过将IBB技术应用于先进的ZOSO方法（即零阶随机方差减小梯度（ZO-SVRG）方法），从而产生了一种方法：ZO-SVRG-IBB，从而展示了IBB技术的功效。我们在理论上分别分析了随机梯度估计器和坐标梯度估计器设置下的ZO-SVRG-IBB方法的收敛性，并表明ZO-SVRG-IBB的收敛率与最先进的高级收敛率相匹配。非凸函数上的ZOSO方法。我们进一步表明，ZO-SVRG-IBB的查询复杂度可与高级ZOSO方法媲美。在不同的数据集上进行的大量数值实验表明，所提出的方法优于或匹配最新的ZOSO方法。