skip to main content
research-article

Scalable Estimator for Multi-task Gaussian Graphical Models Based in an IoT Network

Authors Info & Claims
Published:21 June 2021Publication History
Skip Abstract Section

Abstract

Recently, the Internet of Things (IoT) receives significant interest due to its rapid development. But IoT applications still face two challenges: heterogeneity and large scale of IoT data. Therefore, how to efficiently integrate and process these complicated data becomes an essential problem. In this article, we focus on the problem that analyzing variable dependencies of data collected from different edge devices in the IoT network. Because data from different devices are heterogeneous and the variable dependencies can be characterized into a graphical model, we can focus on the problem that jointly estimating multiple, high-dimensional, and sparse Gaussian Graphical Models for many related tasks (edge devices). This is an important goal in many fields. Many IoT networks have collected massive multi-task data and require the analysis of heterogeneous data in many scenarios. Past works on the joint estimation are non-distributed and involve computationally expensive and complex non-smooth optimizations. To address these problems, we propose a novel approach: Multi-FST. Multi-FST can be efficiently implemented on a cloud-server-based IoT network. The cloud server has a low computational load and IoT devices use asynchronous communication with the server, leading to efficiency. Multi-FST shows significant improvement, over baselines, when tested on various datasets.

References

  1. Eshrat Arjomandi, Michael J. Fischer, and Nancy A. Lynch. 1983. Efficiency of synchronous versus asynchronous distributed systems. J. ACM 30, 3 (1983), 449–456. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Luigi Atzori, Antonio Iera, and Giacomo Morabito. 2010. The internet of things: A survey. Comput. Netw. 54, 15 (2010), 2787–2805. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Onureena Banerjee, Laurent El Ghaoui, and Alexandre d’Aspremont. 2008. Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. Res. 9(Mar.2008), 485–516. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Mohamed Ben-Daya, Elkafi Hassini, and Zied Bahroun. 2019. Internet of things and supply chain management: A literature review. Int. J. Prod. Res. 57, 15–16 (2019), 4719–4742.Google ScholarGoogle ScholarCross RefCross Ref
  5. Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1 (2011), 1–122. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Hongming Cai, Boyi Xu, Lihong Jiang, and Athanasios V. Vasilakos. 2016. IoT-based big data storage systems in cloud computing: Perspectives and challenges. IEEE IoT J. 4, 1 (2016), 75–87.Google ScholarGoogle Scholar
  7. Tony Cai and Weidong Liu. 2011. Adaptive thresholding for sparse covariance matrix estimation. J. Am. Stat. Assoc. 106, 494 (2011), 672–684.Google ScholarGoogle ScholarCross RefCross Ref
  8. Tony Cai, Weidong Liu, and Xi Luo. 2011. A constrained minimization approach to sparse precision matrix estimation. J. Am. Stat. Assoc. 106, 494 (2011), 594–607.Google ScholarGoogle ScholarCross RefCross Ref
  9. Emmanuel Candes, Terence Tao, et al. 2007. The dantzig selector: Statistical estimation when p is much larger than n. Ann. Stat. 35, 6 (2007), 2313–2351.Google ScholarGoogle ScholarCross RefCross Ref
  10. K. Mani Chandy and Jayadev Misra. 1981. Asynchronous distributed simulation via a sequence of parallel computations. Commun. ACM 24, 4 (1981), 198–206. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Tsung-Hui Chang, Mingyi Hong, Wei-Cheng Liao, and Xiangfeng Wang. 2016. Asynchronous distributed ADMM for large-scale optimization—Part I: Algorithm and convergence analysis. IEEE Trans. Sign. Process. 64, 12 (2016), 3118–3130. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Julien Chiquet, Yves Grandvalet, and Christophe Ambroise. 2011. Inferring multiple graphical structures. Stat. Comput. 21, 4 (2011), 537–553. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Dondapati Chowdary, Jessica Lathrop, Joanne Skelton, Kathleen Curtin, Thomas Briggs, Yi Zhang, Jack Yu, Yixin Wang, and Abhijit Mazumder. 2006. Prognostic gene expression signatures can be measured in tissues collected in RNAlater preservative. J. Molec. Diagnost. 8, 1 (2006), 31–39.Google ScholarGoogle ScholarCross RefCross Ref
  14. ENCODE Project Consortium et al. 2004. The ENCODE (ENCyclopedia of DNA elements) project. Science 306, 5696 (2004), 636–640.Google ScholarGoogle Scholar
  15. Flaviu Cristian and Christof Fetzer. 1999. The timed asynchronous distributed system model. IEEE Trans. Parallel Distrib. Syst. 10, 6 (1999), 642–657. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Patrick Danaher, Pei Wang, and Daniela M. Witten. 2014. The joint graphical lasso for inverse covariance estimation across multiple classes. J. Roy. Stat. Soc.: Ser. B Stat. Methodol. 76, 2 (2014), 373–397.Google ScholarGoogle ScholarCross RefCross Ref
  17. Adriana Di Martino, Chao-Gan Yan, Qingyang Li, Erin Denio, Francisco X. Castellanos, Kaat Alaerts, Jeffrey S. Anderson, Michal Assaf, Susan Y. Bookheimer, Mirella Dapretto, et al. 2014. The autism brain imaging data exchange: Towards a large-scale evaluation of the intrinsic brain architecture in autism. Molec. Psychiatr. 19, 6 (2014), 659–667.Google ScholarGoogle ScholarCross RefCross Ref
  18. Adrian Dobra, Chris Hans, Beatrix Jones, Joseph R. Nevins, Guang Yao, and Mike West. 2004. Sparse graphical models for exploring gene expression data. J. Multivar. Anal. 90, 1 (2004), 196–212. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Jianqing Fan and Runze Li. 2001. Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96, 456 (2001), 1348–1360.Google ScholarGoogle ScholarCross RefCross Ref
  20. Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2008. Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9, 3 (2008), 432–441.Google ScholarGoogle ScholarCross RefCross Ref
  21. Xu Gao, Weining Shen, Chee-Ming Ting, Steven C. Cramer, Ramesh Srinivasan, and Hernando Ombao. 2018. Modeling brain connectivity with graphical models on frequency domain. arXiv:1810.03279. Retrieved from https://arxiv.org/abs/1810.03279.Google ScholarGoogle Scholar
  22. Patricia Gonzalez-Guerrero, Stephen G. Wilson, and Mircea R. Stan. 2019. Error-latency trade-off for asynchronous stochastic computing with streams for the IoT. In Proceedings of the 2019 32nd IEEE International System-on-Chip Conference (SOCC’19). IEEE, 97–102.Google ScholarGoogle Scholar
  23. Jian Guo, Elizaveta Levina, George Michailidis, and Ji Zhu. 2011. Joint estimation of multiple graphical models. Biometrika 98, 1 (2011), 1–15.Google ScholarGoogle ScholarCross RefCross Ref
  24. Satoshi Hara and Takashi Washio. 2013. Learning a common substructure of multiple graphical gaussian models. Neur. Netw. 38 (2013), 23–38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Holger Hoefling. 2010. A path algorithm for the fused lasso signal approximator. J. Comput. Graph. Stat. 19, 4 (2010), 984–1006.Google ScholarGoogle ScholarCross RefCross Ref
  26. Jean Honorio and Dimitris Samaras. 2010. Multi-task learning of Gaussian graphical models. In Proceedings of the 27th International Conference on Machine Learning (ICML'10). 447--454. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. John E. Hopcroft and Jeffrey D. Ullman. 1973. Set merging algorithms. SIAM J. Comput. 2, 4 (1973), 294–303.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Wuyungerile Li, Bing Jia, Haotian Xu, Zhaopeng Zong, and Takashi Watanabe. 2019. A multi-task scheduling mechanism based on ACO for maximizing workers’ benefits in mobile crowdsensing service markets with the internet of things. IEEE Access 7 (2019), 41463–41469.Google ScholarGoogle ScholarCross RefCross Ref
  29. Xiaoyuan Liu, Hongwei Li, Guowen Xu, Sen Liu, Zhe Liu, and Rongxing Lu. 2020. PADL: Privacy-aware and asynchronous deep learning for IoT applications. IEEE IoT J. (2020).Google ScholarGoogle Scholar
  30. Meng Ma, Ping Wang, and Chao-Hsien Chu. 2013. Data management for internet of things: Challenges, approaches and opportunities. In Proceedings of the 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing. IEEE, 1144–1151. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Rahul Mazumder and Trevor Hastie. 2012. Exact covariance thresholding into connected components for large-scale graphical lasso. J. Mach. Learn. Res. 13 (Mar. 2012), 781–794. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Tom M. Mitchell, Svetlana V. Shinkareva, Andrew Carlson, Kai-Min Chang, Vicente L. Malave, Robert A. Mason, and Marcel Adam Just. 2008. Predicting human brain activity associated with the meanings of nouns. Science 320, 5880 (2008), 1191–1195.Google ScholarGoogle Scholar
  33. Karthik Mohan, Maryam Fazel Palma London, Daniela Witten, and Su-In Lee. 2014. Node-based learning of multiple Gaussian graphical models. J. Mach. Learn. Res. 15, 1 (2014), 445. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Di Mu, Yunpeng Ge, Mo Sha, Steve Paul, Niranjan Ravichandra, and Souma Chowdhury. 2019. Robust optimal selection of radio type and transmission power for internet of things. ACM Trans. Sen. Netw. 15, 4, Article 39 (July 2019), 25 pages. DOI:https://doi.org/10.1145/3342516 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Michael A. Osborne, Stephen J. Roberts, Alex Rogers, and Nicholas R. Jennings. 2012. Real-time information processing of environmental sensor network data using bayesian gaussian processes. ACM Trans. Sen. Netw. 9, 1, Article 1 (Nov. 2012), 32 pages. DOI:https://doi.org/10.1145/2379799.2379800 Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Rodrigo Perin, Martin Telefont, and Henry Markram. 2013. Computing the size and number of neuronal clusters in local circuits. Front. Neuroanat. 7 (2013), 1.Google ScholarGoogle ScholarCross RefCross Ref
  37. Russell A. Poldrack and Krzysztof J. Gorgolewski. 2017. OpenfMRI: Open sharing of task fMRI data. NeuroImage 144 (2017), 259–261.Google ScholarGoogle ScholarCross RefCross Ref
  38. Adam J. Rothman, Peter J. Bickel, Elizaveta Levina, Ji Zhu, et al. 2008. Sparse permutation invariant covariance estimation. Electr. J. Stat. 2 (2008), 494–515.Google ScholarGoogle ScholarCross RefCross Ref
  39. Adam J. Rothman, Elizaveta Levina, and Ji Zhu. 2009. Generalized thresholding of large covariance matrices. J. Am. Stat. Assoc. 104, 485 (2009), 177–186.Google ScholarGoogle ScholarCross RefCross Ref
  40. Mehrdad Salimitari, Shameek Bhattacharjee, Mainak Chatterjee, and Yaser P. Fallah. 2020. A prospect theoretic approach for trust management in IoT networks under manipulation attacks. ACM Trans. Sens. Netw. 16, 3, Article 26 (May 2020), 26 pages. DOI:https://doi.org/10.1145/3392058 Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Paul T. Schultz and Robert A. Sartini. 2017. IoT communication utilizing secure asynchronous P2P communication and data exchange. US Patent 9,838,204.Google ScholarGoogle Scholar
  42. Shihao Shen, Yiwen Han, Xiaofei Wang, and Yan Wang. 2019. Computation offloading with multiple agents in edge-computing–supported IoT. ACM Trans. Sens. Netw. 16, 1, Article 8 (Dec. 2019), 27 pages. DOI:https://doi.org/10.1145/3372025 Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Noah Simon, Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2013. A sparse-group lasso. J. Comput. Graph. Stat. 22, 2 (2013), 231–245.Google ScholarGoogle ScholarCross RefCross Ref
  44. Zhiyong Sun, Junyong Ye, TongqingWang, Shijian Huang, and Jin Luo. 2020. Behavioral feature recognition of multi-task compressed sensing with fusion relevance in the Internet of Things environment. Comput. Commun, 157 (2020), 381--393.Google ScholarGoogle ScholarCross RefCross Ref
  45. Erming Tian, Fenghuang Zhan, Ronald Walker, Erik Rasmussen, Yupo Ma, Bart Barlogie, and John D Shaughnessy Jr. 2003. The role of the Wnt-signaling antagonist DKK1 in the development of osteolytic lesions in multiple myeloma. N. Engl. J. Med. 349, 26 (2003), 2483–2494.Google ScholarGoogle ScholarCross RefCross Ref
  46. Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. Ser. B Methodol. 58, 1 (1996), 267–288.Google ScholarGoogle ScholarCross RefCross Ref
  47. Ivy F. Tso, Saige Rutherford, Yu Fang, Mike Angstadt, and Stephan F. Taylor. 2018. The “social brain” is highly sensitive to the mere presence of social information: An automated meta-analysis and an independent study. PLoS One 13, 5 (2018).Google ScholarGoogle Scholar
  48. Beilun Wang, Ji Gao, and Yanjun Qi. 2017. A fast and scalable joint estimator for learning multiple related sparse gaussian graphical models. arXiv:1702.02715. Retrieved from https://arxiv.org/abs/1702.02715.Google ScholarGoogle Scholar
  49. Beilun Wang, Ritambhara Singh, and Yanjun Qi. 2017. A constrained l1 minimization approach for estimating multiple sparse Gaussian or nonparanormal graphical models. Mach. Learn. 106, 9-10 (2017), 1381–1417. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Lizhe Wang and Rajiv Ranjan. 2015. Processing distributed internet of things data in clouds. IEEE Cloud Comput. 2, 1 (2015), 76–80.Google ScholarGoogle ScholarCross RefCross Ref
  51. Daniela M. Witten, Jerome H. Friedman, and Noah Simon. 2011. New insights and faster computations for the graphical lasso. J. Comput. Graph. Stat. 20, 4 (2011), 892–900.Google ScholarGoogle ScholarCross RefCross Ref
  52. Nuosi Wu, Jiang Huang, Xiao-Fei Zhang, Le Ou-Yang, Shan He, Zexuan Zhu, and Weixin Xie. 2019. Weighted fused pathway graphical lasso for joint estimation of multiple gene networks. Front. Genet. 10 (2019), 623.Google ScholarGoogle ScholarCross RefCross Ref
  53. Han Xiao, Changqiao Xu, Tengfei Cao, Lujie Zhong, and Gabriel-Miro Muntean. 2019. GTTC: A low-expenditure IoT multi-task coordinated distributed computing framework with fog computing. In Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM’19). IEEE, 1–6.Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Eunho Yang, Aurélie C. Lozano, and Pradeep K. Ravikumar. 2014. Elementary estimators for graphical models. In Advances in Neural Information Processing Systems. 2159–2167. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Ming Yuan and Yi Lin. 2007. Model selection and estimation in the gaussian graphical model. Biometrika 94, 1 (2007), 19–35.Google ScholarGoogle ScholarCross RefCross Ref
  56. Bai Zhang and Yue Wang. 2012. Learning structural changes of Gaussian graphical models in controlled experiments. arXiv:1203.3532. Retrieved from https://arxiv.org/abs/1203.3532. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. L. Zhang, S. Yu, X. Ding, and X. Wang. 2014. Research on IOT RESTful web service asynchronous composition based on BPEL. In Proceedings of the 2014 6th International Conference on Intelligent Human-Machine Systems and Cybernetics, Vol. 1. 62–65. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Yi Zhang and Jeff G. Schneider. 2010. Learning multiple tasks with a sparse matrix-normal penalty. In Advances in Neural Information Processing Systems. 2550–2558. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Yunzhang Zhu, Xiaotong Shen, and Wei Pan. 2014. Structural pursuit over multiple undirected graphs. J. Am. Stat. Assoc. 109, 508 (2014), 1683–1696.Google ScholarGoogle ScholarCross RefCross Ref
  60. Hui Zou. 2006. The adaptive lasso and its oracle properties. J. Am. Stat. Assoc. 101, 476 (2006), 1418–1429.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Scalable Estimator for Multi-task Gaussian Graphical Models Based in an IoT Network

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Sensor Networks
          ACM Transactions on Sensor Networks  Volume 17, Issue 3
          August 2021
          333 pages
          ISSN:1550-4859
          EISSN:1550-4867
          DOI:10.1145/3470624
          Issue’s Table of Contents

          Copyright © 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM.

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 21 June 2021
          • Accepted: 1 October 2020
          • Revised: 1 September 2020
          • Received: 1 July 2020
          Published in tosn Volume 17, Issue 3

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format