Decentralized stochastic gradient methods play significant roles in large-scale optimization that finds many practical applications in machine learning and coordinated control. This paper studies optimization problems over unbalanced directed networks, where the mutual goal of agents in the network is to optimize a global objective function expressed as a sum of local objective functions. Each agent using only local computation and communication in the networks is assumed to get access to a stochastic first-order oracle. In order to devise a noise-tolerant decentralized algorithm with accelerated linear convergence, a decentralized Nesterov gradient algorithm with the constant step-size and parameter using stochastic gradients is proposed in this paper. The proposed algorithm employing a gradient-tracking technique is proved to converge linearly to an error ball around the optimal solution via the analysis on a linear system when the positive constant step-size and parameter are sufficiently small. We further recover the exact linear convergence for the proposed algorithm with exact gradients under the same selection conditions of the constant step-size and parameter. Some real-world data sets are used in simulations to validate the correctness of the theoretical findings and practicability of the proposed algorithm.

中文翻译：

---

不平衡有向网络随机优化的分散 Nesterov 梯度方法

分散随机梯度方法在大规模优化中发挥着重要作用，在机器学习和协调控制中发现了许多实际应用。本文研究了不平衡有向网络上的优化问题，其中网络中代理的共同目标是优化表示为局部目标函数之和的全局目标函数。假设每个仅使用网络中本地计算和通信的代理都可以访问随机一阶预言机。为了设计一种具有加速线性收敛的去中心化算法，本文提出了一种使用随机梯度的具有恒定步长和参数的去中心化Nesterov梯度算法。通过对线性系统的分析，证明了采用梯度跟踪技术的算法在正常数步长和参数足够小时，可以线性收敛到最优解周围的误差球。在恒定步长和参数的相同选择条件下，我们进一步恢复了具有精确梯度的所提出算法的精确线性收敛。一些真实世界的数据集用于模拟，以验证理论发现的正确性和所提出算法的实用性。在恒定步长和参数的相同选择条件下，我们进一步恢复了具有精确梯度的所提出算法的精确线性收敛。一些真实世界的数据集用于模拟，以验证理论发现的正确性和所提出算法的实用性。在恒定步长和参数的相同选择条件下，我们进一步恢复了具有精确梯度的所提出算法的精确线性收敛。一些真实世界的数据集用于模拟，以验证理论发现的正确性和所提出算法的实用性。