We consider learning in decentralized heterogeneous networks: agents seek to minimize a convex functional that aggregates data across the network, while only having access to their local data streams. We focus on the case where agents seek to estimate a regression function that belongs to a reproducing kernel Hilbert space (RKHS). To incentivize coordination while respecting network heterogeneity, we impose nonlinear proximity constraints. The resulting constrained stochastic optimization problem is solved using the functional variant of stochastic primal-dual (Arrow-Hurwicz) method which yields a decentralized algorithm. In order to avoid the model complexity from growing linearly over time, we project the primal iterates onto subspaces greedily constructed from kernel evaluations of agents’ local observations. The resulting scheme, dubbed Heterogeneous Adaptive Learning with Kernels (HALK), allows us, for the first time, to characterize the precise trade-off between the optimality gap, constraint violation, and the model complexity. In particular, the proposed algorithm can be tuned to achieve zero constraint violation, an optimality gap of ${\mathcal O}(T^{-1/2}+\alpha)$ after $T$ iterations, where the number of elements retained in the dictionary is determined by $1/\alpha$ . Simulations on a correlated spatio-temporal field estimation validate our theoretical results, which are corroborated in practice for networked oceanic sensing buoys estimating temperature and salinity from depth measurements.

中文翻译：

---

异构网络中的自适应内核学习

我们考虑在去中心化异构网络中学习：代理试图最小化在网络上聚合数据的凸函数，同时只能访问其本地数据流。我们专注于代理试图估计回归的情况属于再生核希尔伯特空间 (RKHS) 的函数。为了在尊重网络异构性的同时激励协调，我们施加了非线性邻近约束。使用随机原始对偶 (Arrow-Hurwicz) 方法的函数变体解决了由此产生的约束随机优化问题，该方法产生了一种分散算法。为了避免模型复杂性随时间线性增长，我们将原始迭代投影到从代理局部观察的内核评估贪婪地构建的子空间上。由此产生的方案被称为具有内核的异构自适应学习 (HALK)，使我们第一次能够表征最优性差距、约束违反和模型复杂性之间的精确权衡。特别是，${\mathcal O}(T^{-1/2}+\alpha)$ 后 $T$ 迭代，其中保留在字典中的元素数量由 $1/\alpha$ . 对相关时空场估计的模拟验证了我们的理论结果，这些结果在实践中得到了证实，用于网络海洋传感浮标从深度测量中估计温度和盐度。