We seek tight bounds on the viable parallelism in asynchronous implementations of coordinate descent that achieves linear speedup. We focus on asynchronous coordinate descent (ACD) algorithms on convex functions which consist of the sum of a smooth convex part and a possibly non-smooth separable convex part. We quantify the shortfall in progress compared to the standard sequential stochastic gradient descent. This leads to a simple yet tight analysis of the standard stochastic ACD in a partially asynchronous environment, generalizing and improving the bounds in prior work. We also give a considerably more involved analysis for general asynchronous environments in which the only constraint is that each update can overlap with at most q others. The new lower bound on the maximum degree of parallelism attaining linear speedup is tight and improves the best prior bound almost quadratically.

中文翻译：

---

完全异步随机坐标下降：实现线性加速的并行度的严格下限

我们在实现线性加速的坐标下降的异步实现中寻求可行并行性的严格界限。我们专注于凸函数的异步坐标下降 (ACD) 算法，该算法由平滑的凸部分和可能的非平滑可分离凸部分的总和组成。与标准顺序随机梯度下降相比，我们量化了进展中的不足。这导致在部分异步环境中对标准随机 ACD 进行简单而严格的分析，概括和改进先前工作中的界限。我们还对一般异步环境进行了相当多的分析，其中唯一的限制是每个更新最多可以与其他 q 个更新重叠。