Relative smoothness—a notion introduced in Birnbaum et al. (Proceedings of the 12th ACM conference on electronic commerce, ACM, pp 127–136, 2011) and recently rediscovered in Bauschke et al. (Math Oper Res 330–348, 2016) and Lu et al. (Relatively-smooth convex optimization by first-order methods, and applications, arXiv:1610.05708, 2016)—generalizes the standard notion of smoothness typically used in the analysis of gradient type methods. In this work we are taking ideas from well studied field of stochastic convex optimization and using them in order to obtain faster algorithms for minimizing relatively smooth functions. We propose and analyze two new algorithms: Relative Randomized Coordinate Descent (relRCD) and Relative Stochastic Gradient Descent (relSGD), both generalizing famous algorithms in the standard smooth setting. The methods we propose can be in fact seen as particular instances of stochastic mirror descent algorithms, which has been usually analyzed under stronger assumptions: Lipschitzness of the objective and strong convexity of the reference function. As a consequence, one of the proposed methods, relRCD corresponds to the first stochastic variant of mirror descent algorithm with linear convergence rate.

相对平滑度——Birnbaum 等人中引入的一个概念。（第 12 届 ACM 电子商务会议论文集，ACM，第 127-136 页，2011 年），最近在 Bauschke 等人中重新发现。(Math Oper Res 330–348, 2016) 和 Lu 等人。（通过一阶方法和应用程序进行相对平滑的凸优化，arXiv:1610.05708, 2016）——概括了通常用于梯度类型方法分析的平滑度标准概念。在这项工作中，我们从随机凸优化的研究领域中汲取了一些想法，并使用它们来获得更快的算法来最小化相对平滑的函数。我们提出并分析了两种新算法：相对随机坐标下降 (relRCD) 和相对随机梯度下降 (relSGD)，它们都在标准平滑设置中推广了著名的算法。我们提出的方法实际上可以看作是随机镜像下降算法的特定实例，通常在更强的假设下进行分析：目标的 Lipschitzness 和参考函数的强凸性。因此，所提出的方法之一 relRCD 对应于具有线性收敛率的镜像下降算法的第一个随机变体。