We present a convergence rate analysis for biased stochastic gradient descent (SGD), where individual gradient updates are corrupted by computation errors. We develop stochastic quadratic constraints to formulate a small linear matrix inequality (LMI) whose feasible points lead to convergence bounds of biased SGD. Based on this LMI condition, we develop a sequential minimization approach to analyze the intricate trade-offs that couple stepsize selection, convergence rate, optimization accuracy, and robustness to gradient inaccuracy. We also provide feasible points for this LMI and obtain theoretical formulas that quantify the convergence properties of biased SGD under various assumptions on the loss functions.

中文翻译：

---

使用顺序半定程序分析有偏随机梯度下降

我们提出了有偏随机梯度下降 (SGD) 的收敛率分析，其中单个梯度更新被计算错误破坏。我们开发了随机二次约束来制定一个小的线性矩阵不等式 (LMI)，其可行点导致有偏 SGD 的收敛边界。基于此 LMI 条件，我们开发了一种顺序最小化方法来分析复杂的权衡，这些权衡将步长选择、收敛速度、优化精度和梯度不准确性的鲁棒性结合起来。我们还为此 LMI 提供了可行的点，并获得了在损失函数的各种假设下量化有偏 SGD 收敛特性的理论公式。