Many machine learning algorithms have been developed under the assumption that datasets are already available in batch form. Yet, in many application domains, data are only available sequentially overtime via compute nodes in different geographic locations. In this article, we consider the problem of learning a model when streaming data cannot be transferred to a single location in a timely fashion. In such cases, a distributed architecture for learning which relies on a network of interconnected “local” nodes is required. We propose a distributed scheme in which every local node implements stochastic gradient updates based upon a local data stream. To ensure robust estimation, a network regularization penalty is used to maintain a measure of cohesion in the ensemble of models. We show that the ensemble average approximates a stationary point and characterizes the degree to which individual models differ from the ensemble average. We compare the results with federated learning to conclude that the proposed approach is more robust to heterogeneity in data streams (data rates and estimation quality). We illustrate the results with an application to image classification with a deep learning model based upon convolutional neural networks.

中文翻译：

---

分布式网络实时学习

在假设数据集已经以批处理形式可用的前提下，已经开发了许多机器学习算法。但是，在许多应用程序域中，只有通过不同地理位置的计算节点才能按时间顺序顺序获得数据。在本文中，我们考虑了无法将流数据及时传输到单个位置时学习模型的问题。在这种情况下，需要依赖于互连的“本地”节点网络的分布式学习体系结构。我们提出了一种分布式方案，其中每个本地节点都基于本地数据流实现随机梯度更新。为了确保可靠的估计，网络正则化罚分用于在模型集合中保持对内聚度的度量。我们表明，集合平均数近似于一个固定点，并描述了各个模型与集合平均数的差异程度。我们将结果与联合学习进行了比较，得出的结论是，所提出的方法对于数据流（数据速率和估计质量）中的异构性更加鲁棒。我们通过基于卷积神经网络的深度学习模型将其应用于图像分类来说明结果。