Federated learning (FL) and split learning (SL) are state-of-the-art distributed machine learning techniques to enable machine learning training without accessing raw data on clients or end devices. However, their \emph{comparative training performance} under real-world resource-restricted Internet of Things (IoT) device settings, e.g., Raspberry Pi, remains barely studied, which, to our knowledge, have not yet been evaluated and compared, rendering inconvenient reference for practitioners. This work firstly provides empirical comparisons of FL and SL in real-world IoT settings regarding (i) learning performance with heterogeneous data distributions and (ii) on-device execution overhead. Our analyses in this work demonstrate that the learning performance of SL is better than FL under an imbalanced data distribution but worse than FL under an extreme non-IID data distribution. Recently, FL and SL are combined to form splitfed learning (SFL) to leverage each of their benefits (e.g., parallel training of FL and lightweight on-device computation requirement of SL). This work then considers FL, SL, and SFL, and mount them on Raspberry Pi devices to evaluate their performance, including training time, communication overhead, power consumption, and memory usage. Besides evaluations, we apply two optimizations. Firstly, we generalize SFL by carefully examining the possibility of a hybrid type of model training at the server-side. The generalized SFL merges sequential (dependent) and parallel (independent) processes of model training and is thus beneficial for a system with large-scaled IoT devices, specifically at the server-side operations. Secondly, we propose pragmatic techniques to substantially reduce the communication overhead by up to four times for the SL and (generalized) SFL.

中文翻译：

---

物联网分布式机器学习技术的评估与优化

联合学习（FL）和拆分学习（SL）是最先进的分布式机器学习技术，可在不访问客户端或终端设备上的原始数据的情况下进行机器学习训练。但是，他们在真实资源受限的物联网（IoT）设备设置（例如Raspberry Pi）下的\ emph {比较训练效果}仍很少得到研究，据我们所知，尚未对其进行评估和比较。给从业者带来不便。这项工作首先提供了在现实物联网设置中FL和SL的经验比较，涉及（i）具有异构数据分布的学习性能和（ii）设备执行开销。我们在这项工作中的分析表明，在不平衡的数据分布下，SL的学习性能优于FL，但在极端的非IID数据分布下，则比FL差。最近，FL和SL结合在一起形成了组合学习（SFL），以利用它们的每个好处（例如，FL的并行训练和SL的轻型设备上计算要求）。然后，本文将考虑FL，SL和SFL，并将它们安装在Raspberry Pi设备上以评估其性能，包括培训时间，通信开销，功耗和内存使用情况。除了评估外，我们还应用了两个优化。首先，我们通过仔细检查服务器端混合类型的模型训练的可能性来概括SFL。通用SFL融合了模型训练的顺序（相关）和并行（独立）过程，因此对于具有大规模IoT设备的系统（特别是在服务器端操作）有利。其次，我们提出了实用的技术，可将SL和（通用）SFL的通信开销大幅度减少多达四倍。