Skip to main content
SpringerLink
Log in

Adaptive Clustered Federated Learning for Heterogeneous Data in Edge Computing

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

Although federated learning has been widely used in collaborative training of machine learning models, its practical uses are still challenged by heterogeneous data across clients. To alleviate the impact of non-IID data issue, we present an adaptive clustered federated learning approach, \(\mathtt {AdaCFL}\), which can classify clients into suitable clusters according to their local data distribution and train a specialized model for the clients of each cluster. By exploiting the implicit connection between local model weights and data distribution on clients, \(\mathtt {AdaCFL}\) relies on partial selected model weights to measure the data similarity between clients and adaptively groups them into the optimal number of clusters. Experimental results on three benchmark datasets with various non-IID data settings demonstrate that \(\mathtt {AdaCFL}\) achieves comparably high model accuracy as the state-of-the-art works, yet with a significant reduction on the communication cost.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availability

The datasets i.e., MNIST [11], CIFAR-10 [10] and FashionMNIST [8], analysed during the current study are available from http://yann.lecun.com/exdb/mnist/, http://www.cs.toronto.edu/ ~ kriz/cifar.html and https://github.com/zalandoresearch/fashion-mnist

References

  1. Bonawitz K, Eichner H, Grieskamp W, Huba D, Ingerman A, Ivanov V, Kiddon C, Konečnỳ J, Mazzocchi S, McMahan B et al (2019) Towards federated learning at scale: System design. Proceedings of Machine Learning and Systems 1:374–388

    Google Scholar 

  2. Dasgupta S, Long PM (2005) Performance guarantees for hierarchical clustering. J Comput Syst Sci 70(4):555–569

    Article  MathSciNet  Google Scholar 

  3. Duan M, Liu D, Ji X, Liu R, Liang L, Chen X, Tan Y (2020) Fedgroup: Ternary cosine similarity-based clustered federated learning framework toward high accuracy in heterogeneous data. arXiv preprint arXiv:201006870

  4. Gao H, Liu C, Yin Y, Xu Y, Li Y (2021a) A hybrid approach to trust node assessment and management for vanets cooperative data communication: Historical interaction perspective. IEEE Transactions on Intelligent Transportation Systems

  5. Gao H, Yin Y, Han G, Zhao W (2021b) Edge computing: Enabling technologies, applications, and services. Transactions on Emerging Telecommunications Technologies 32(6)

  6. Gao H, Zhou L, Kim JY, Li Y, Huang W (2021c) Applying probabilistic model checking to the behavior guidance and abnormality detection for mci patients under wireless sensor network. ACM Transactions on Sensor Networks https://doi.org/10.1145/3499426

  7. Ghosh A, Chung J, Yin D, Ramchandran K (2020) An efficient framework for clustered federated learning. Adv Neural Inf Process Syst 33:19586–19597

    Google Scholar 

  8. Han X, Kashif R, Roland V (2017) The fashionmnist dataset. online: https://github.com/zalandoresearch/fashion-mnist

  9. Hsieh K, Phanishayee A, Mutlu O, Gibbons P (2020) The non-iid data quagmire of decentralized machine learning. In: International Conference on Machine Learning, pp 4387–4398

  10. Krizhevsky A, Hinton G, et al. (2014) The cifar-10 dataset. online: http://www.cstoronto.edu/~kriz/cifar.html

  11. LeCun Y (1998) The mnist database of handwritten digits. online: http://yann lecun com/exdb/mnist/

  12. Li T, Sahu AK, Zaheer M, Sanjabi M, Talwalkar A, Smith V (2020a) Federated optimization in heterogeneous networks. vol 2, pp 429–450

  13. Li X, Huang K, Yang W, Wang S, Zhang Z (2020b) On the convergence of fedavg on non-iid data. In: International Conference on Learning Representations

  14. Liang T, Sheng X, Zhou L, Li Y, Gao H, Yin Y, Chen L (2021) Mobile app recommendation via heterogeneous graph neural network in edge computing. Appl Soft Comput 103:107162

  15. Liu Y, Huang A, Luo Y, Huang H, Liu Y, Chen Y, Feng L, Chen T, Yu H, Yang Q (2020) Fedvision: An online visual object detection platform powered by federated learning. Proceedings of the AAAI Conference on Artificial Intelligence 34:13172–13179

    Article  Google Scholar 

  16. Long M, Cao Y, Cao Z, Wang J, Jordan MI (2018) Transferable representation learning with deep adaptation networks. IEEE Trans Pattern Anal Mach Intell 41(12):3071–3085

    Article  Google Scholar 

  17. Ma X, Xu H, Gao H, Bian M (2021) Real-time multiple-workflow scheduling in cloud environments. IEEE Trans Netw Serv Manage. https://doi.org/10.1109/TNSM.2021.3125395

    Article  Google Scholar 

  18. McMahan B, Moore E, Ramage D, Hampson S, Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics, pp 1273–1282

  19. Mou L, Meng Z, Yan R, Li G, Xu Y, Zhang L, Jin Z (2016) How transferable are neural networks in nlp applications? In: Conference on Empirical Methods in Natural Language Processing, pp 479–489

  20. Ouyang X, Xie Z, Zhou J, Huang J, Xing G (2021) Clusterfl: a similarity-aware federated learning system for human activity recognition. In: Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services, pp 54–66

  21. Sattler F, Wiedemann S, Müller KR, Samek W (2019) Robust and communication-efficient federated learning from non-iid data. IEEE transactions on neural networks and learning systems 31(9):3400–3413

    Article  Google Scholar 

  22. Sattler F, Müller KR, Samek W (2020) Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints. IEEE transactions on neural networks and learning systems 32(8):3710–3722

    Article  MathSciNet  Google Scholar 

  23. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556

  24. Smith V, Chiang CK, Sanjabi M, Talwalkar AS (2017) Federated multi-task learning. vol 30

  25. Wang H, Kaplan Z, Niu D, Li B (2020) Optimizing federated learning on non-iid data with reinforcement learning. In: IEEE Conference on Computer Communications, pp 1698–1707

  26. Wang K, Mathews R, Kiddon C, Eichner H, Beaufays F, Ramage D (2019) Federated evaluation of on-device personalization. arXiv preprint arXiv:191010252

  27. Wang Z, Xu H, Liu J, Huang H, Qiao C, Zhao Y (2021) Resource-efficient federated learning with hierarchical aggregation in edge computing. In: IEEE Conference on Computer Communications, IEEE, pp 1–10

  28. Xie M, Long G, Shen T, Zhou T, Wang X, Jiang J, Zhang C (2021) Multi-center federated learning. arXiv preprint arXiv:210808647

  29. Yang Q, Liu Y, Chen T, Tong Y (2019) Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology 10(2):1–19

    Article  Google Scholar 

  30. Yin Y, Cao Z, Xu Y, Gao H, Li R, Mai Z (2020a) Qos prediction for service recommendation with features learning in mobile edge computing environment. IEEE Transactions on Cognitive Communications and Networking 6(4):1136–1145

  31. Yin Y, Huang Q, Gao H, Xu Y (2020b) Personalized apis recommendation with cognitive knowledge mining for industrial systems. IEEE Transactions on Industrial Informatics

  32. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? vol 27

  33. Zheng W, Yan L, Gou C, Wang FY (2021) Federated meta-learning for fraudulent credit card detection. In: Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pp 4654–4660

  34. Zhu X, Wang J, Hong Z, Xiao J (2020) Empirical studies of institutional federated learning for natural language processing. In: Findings of the Association for Computational Linguistics: EMNLP, pp 625–634

Download references

Acknowledgements

This work was supported in part by International Cooperation Project of Shaanxi Province (No. 2020KW-004), the China Postdoctoral Science Foundation (No. 2017M613187), the Shaanxi Science and Technology Innovation Team Support Project under grant agreement (No. 2018TD-026), the China NSFC Grant (No.62172284) and the Natural Science Foundation of Guangdong (General Program No.2020A1515011502).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Northwest University, Xi’an, 710127, China

    Biyao Gong, Tianzhang Xing, Junfeng Wang & Xiuya Liu

  2. College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China

    Zhidan Liu

Authors
  1. Biyao Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tianzhang Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhidan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Junfeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiuya Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Tianzhang Xing or Zhidan Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gong, B., Xing, T., Liu, Z. et al. Adaptive Clustered Federated Learning for Heterogeneous Data in Edge Computing. Mobile Netw Appl 27, 1520–1530 (2022). https://doi.org/10.1007/s11036-022-01978-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-022-01978-8

Keywords

Search

Navigation