Block-coordinate descent is a popular framework for large-scale regularized optimization problems with block-separable structure. Existing methods have several limitations. They often assume that subproblems can be solved exactly at each iteration, which in practical terms usually restricts the quadratic term in the subproblem to be diagonal, thus losing most of the benefits of higher-order derivative information. Moreover, in contrast to the smooth case, non-uniform sampling of the blocks has not yet been shown to improve the convergence rate bounds for regularized problems. This work proposes an inexact randomized block-coordinate descent method based on a regularized quadratic subproblem, in which the quadratic term can vary from iteration to iteration: a “variable metric.” We provide a detailed convergence analysis for both convex and non-convex problems. Our analysis generalizes, to the regularized case, Nesterov’s proposal for improving convergence of block-coordinate descent by sampling proportional to the blockwise Lipschitz constants. We improve the convergence rate in the convex case by weakening the dependency on the initial objective value. Empirical results also show that significant benefits accrue from the use of a variable metric.

中文翻译：

---

用于正则化优化的不精确变量度量随机块坐标下降

块坐标下降是用于具有块可分离结构的大规模正则化优化问题的流行框架。现有方法有几个限制。他们通常假设子问题可以在每次迭代中精确求解，这在实际中通常将子问题中的二次项限制为对角线，从而失去了高阶导数信息的大部分好处。此外，与平滑情况相比，块的非均匀采样还没有被证明可以提高正则化问题的收敛速度界限。这项工作提出了一种基于正则化二次子问题的不精确随机块坐标下降方法，其中二次项可以随迭代而变化：“可变度量”。”我们为凸和非凸问题提供了详细的收敛分析。我们的分析将 Nesterov 的建议推广到正则化的情况，即通过与块状 Lipschitz 常数成比例的采样来提高块坐标下降的收敛性。我们通过削弱对初始目标值的依赖来提高凸情况下的收敛速度。实证结果还表明，使用可变指标会产生显着的好处。