Applied and Computational Harmonic Analysis ( IF 2.6 ) Pub Date : 2022-03-21 , DOI: 10.1016/j.acha.2022.03.003 HanQin Cai 1 , Daniel McKenzie 1 , Wotao Yin 2 , Zhenliang Zhang 3
We study zeroth-order optimization for convex functions where we further assume that function evaluations are unavailable. Instead, one only has access to a comparison oracle, which given two points x and y returns a single bit of information indicating which point has larger function value, or . By treating the gradient as an unknown signal to be recovered, we show how one can use tools from one-bit compressed sensing to construct a robust and reliable estimator of the normalized gradient. We then propose an algorithm, coined SCOBO, that uses this estimator within a gradient descent scheme. We show that when has some low dimensional structure that can be exploited, SCOBO outperforms the state-of-the-art in terms of query complexity. Our theoretical claims are verified by extensive numerical experiments.
中文翻译:
一位基于比较的梯度估计器
我们研究凸函数的零阶优化,我们进一步假设函数评估不可用。相反,一个人只能访问一个比较 oracle,它给定两个点x和y返回一位信息,指示哪个点具有更大的函数值,要么. 通过将梯度视为要恢复的未知信号,我们展示了如何使用来自一位压缩感知的工具来构建一个稳健可靠的归一化梯度估计器。然后,我们提出了一种算法,称为 SCOBO,它在梯度下降方案中使用这个估计器。我们证明当SCOBO 具有一些可以利用的低维结构,在查询复杂性方面优于最先进的技术。我们的理论主张通过广泛的数值实验得到验证。