SIAM Journal on Optimization, Volume 31, Issue 1, Page 991-1016, January 2021.
The conditions of relative smoothness and relative strong convexity were recently introduced for the analysis of Bregman gradient methods for convex optimization. We introduce a generalized left-preconditioning method for gradient descent and show that its convergence on an essentially smooth convex objective function can be guaranteed via an application of relative smoothness in the dual space. Our relative smoothness assumption is between the designed preconditioner and the convex conjugate of the objective, and it generalizes the typical Lipschitz gradient assumption. Under dual relative strong convexity, we obtain linear convergence with a generalized condition number that is invariant under horizontal translations, distinguishing it from Bregman gradient methods. Thus, in principle our method is capable of improving the conditioning of gradient descent on problems with a non-Lipschitz gradient or nonstrongly convex structure. We demonstrate our method on $p$-norm regression and exponential penalty function minimization.

中文翻译：

---

梯度下降的双重空间预处理

SIAM优化杂志，第31卷，第1期，第991-1016页，2021年1月。
最近介绍了相对光滑度和相对强凸度的条件，以分析用于凸优化的Bregman梯度方法。我们介绍了一种梯度下降的广义左预处理方法，并表明通过在对偶空间中应用相对光滑度，可以保证其在基本光滑的凸目标函数上的收敛性。我们的相对平滑度假设介于设计的预处理器和物镜的凸共轭之间，并推广了典型的Lipschitz梯度假设。在双重相对强凸下，我们获得了线性条件下的线性收敛，广义条件数在水平平移下不变，这与Bregman梯度法是有区别的。因此，原则上，对于非Lipschitz梯度或非强凸结构的问题，我们的方法能够改善梯度下降的条件。我们演示了关于$ p $ -norm回归和指数罚函数最小化的方法。