Abstract
Recently, the Internet of Things (IoT) receives significant interest due to its rapid development. But IoT applications still face two challenges: heterogeneity and large scale of IoT data. Therefore, how to efficiently integrate and process these complicated data becomes an essential problem. In this article, we focus on the problem that analyzing variable dependencies of data collected from different edge devices in the IoT network. Because data from different devices are heterogeneous and the variable dependencies can be characterized into a graphical model, we can focus on the problem that jointly estimating multiple, high-dimensional, and sparse Gaussian Graphical Models for many related tasks (edge devices). This is an important goal in many fields. Many IoT networks have collected massive multi-task data and require the analysis of heterogeneous data in many scenarios. Past works on the joint estimation are non-distributed and involve computationally expensive and complex non-smooth optimizations. To address these problems, we propose a novel approach: Multi-FST. Multi-FST can be efficiently implemented on a cloud-server-based IoT network. The cloud server has a low computational load and IoT devices use asynchronous communication with the server, leading to efficiency. Multi-FST shows significant improvement, over baselines, when tested on various datasets.
- Eshrat Arjomandi, Michael J. Fischer, and Nancy A. Lynch. 1983. Efficiency of synchronous versus asynchronous distributed systems. J. ACM 30, 3 (1983), 449–456. Google ScholarDigital Library
- Luigi Atzori, Antonio Iera, and Giacomo Morabito. 2010. The internet of things: A survey. Comput. Netw. 54, 15 (2010), 2787–2805. Google ScholarDigital Library
- Onureena Banerjee, Laurent El Ghaoui, and Alexandre d’Aspremont. 2008. Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. Res. 9(Mar.2008), 485–516. Google ScholarDigital Library
- Mohamed Ben-Daya, Elkafi Hassini, and Zied Bahroun. 2019. Internet of things and supply chain management: A literature review. Int. J. Prod. Res. 57, 15–16 (2019), 4719–4742.Google ScholarCross Ref
- Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1 (2011), 1–122. Google ScholarDigital Library
- Hongming Cai, Boyi Xu, Lihong Jiang, and Athanasios V. Vasilakos. 2016. IoT-based big data storage systems in cloud computing: Perspectives and challenges. IEEE IoT J. 4, 1 (2016), 75–87.Google Scholar
- Tony Cai and Weidong Liu. 2011. Adaptive thresholding for sparse covariance matrix estimation. J. Am. Stat. Assoc. 106, 494 (2011), 672–684.Google ScholarCross Ref
- Tony Cai, Weidong Liu, and Xi Luo. 2011. A constrained minimization approach to sparse precision matrix estimation. J. Am. Stat. Assoc. 106, 494 (2011), 594–607.Google ScholarCross Ref
- Emmanuel Candes, Terence Tao, et al. 2007. The dantzig selector: Statistical estimation when p is much larger than n. Ann. Stat. 35, 6 (2007), 2313–2351.Google ScholarCross Ref
- K. Mani Chandy and Jayadev Misra. 1981. Asynchronous distributed simulation via a sequence of parallel computations. Commun. ACM 24, 4 (1981), 198–206. Google ScholarDigital Library
- Tsung-Hui Chang, Mingyi Hong, Wei-Cheng Liao, and Xiangfeng Wang. 2016. Asynchronous distributed ADMM for large-scale optimization—Part I: Algorithm and convergence analysis. IEEE Trans. Sign. Process. 64, 12 (2016), 3118–3130. Google ScholarDigital Library
- Julien Chiquet, Yves Grandvalet, and Christophe Ambroise. 2011. Inferring multiple graphical structures. Stat. Comput. 21, 4 (2011), 537–553. Google ScholarDigital Library
- Dondapati Chowdary, Jessica Lathrop, Joanne Skelton, Kathleen Curtin, Thomas Briggs, Yi Zhang, Jack Yu, Yixin Wang, and Abhijit Mazumder. 2006. Prognostic gene expression signatures can be measured in tissues collected in RNAlater preservative. J. Molec. Diagnost. 8, 1 (2006), 31–39.Google ScholarCross Ref
- ENCODE Project Consortium et al. 2004. The ENCODE (ENCyclopedia of DNA elements) project. Science 306, 5696 (2004), 636–640.Google Scholar
- Flaviu Cristian and Christof Fetzer. 1999. The timed asynchronous distributed system model. IEEE Trans. Parallel Distrib. Syst. 10, 6 (1999), 642–657. Google ScholarDigital Library
- Patrick Danaher, Pei Wang, and Daniela M. Witten. 2014. The joint graphical lasso for inverse covariance estimation across multiple classes. J. Roy. Stat. Soc.: Ser. B Stat. Methodol. 76, 2 (2014), 373–397.Google ScholarCross Ref
- Adriana Di Martino, Chao-Gan Yan, Qingyang Li, Erin Denio, Francisco X. Castellanos, Kaat Alaerts, Jeffrey S. Anderson, Michal Assaf, Susan Y. Bookheimer, Mirella Dapretto, et al. 2014. The autism brain imaging data exchange: Towards a large-scale evaluation of the intrinsic brain architecture in autism. Molec. Psychiatr. 19, 6 (2014), 659–667.Google ScholarCross Ref
- Adrian Dobra, Chris Hans, Beatrix Jones, Joseph R. Nevins, Guang Yao, and Mike West. 2004. Sparse graphical models for exploring gene expression data. J. Multivar. Anal. 90, 1 (2004), 196–212. Google ScholarDigital Library
- Jianqing Fan and Runze Li. 2001. Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96, 456 (2001), 1348–1360.Google ScholarCross Ref
- Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2008. Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9, 3 (2008), 432–441.Google ScholarCross Ref
- Xu Gao, Weining Shen, Chee-Ming Ting, Steven C. Cramer, Ramesh Srinivasan, and Hernando Ombao. 2018. Modeling brain connectivity with graphical models on frequency domain. arXiv:1810.03279. Retrieved from https://arxiv.org/abs/1810.03279.Google Scholar
- Patricia Gonzalez-Guerrero, Stephen G. Wilson, and Mircea R. Stan. 2019. Error-latency trade-off for asynchronous stochastic computing with streams for the IoT. In Proceedings of the 2019 32nd IEEE International System-on-Chip Conference (SOCC’19). IEEE, 97–102.Google Scholar
- Jian Guo, Elizaveta Levina, George Michailidis, and Ji Zhu. 2011. Joint estimation of multiple graphical models. Biometrika 98, 1 (2011), 1–15.Google ScholarCross Ref
- Satoshi Hara and Takashi Washio. 2013. Learning a common substructure of multiple graphical gaussian models. Neur. Netw. 38 (2013), 23–38. Google ScholarDigital Library
- Holger Hoefling. 2010. A path algorithm for the fused lasso signal approximator. J. Comput. Graph. Stat. 19, 4 (2010), 984–1006.Google ScholarCross Ref
- Jean Honorio and Dimitris Samaras. 2010. Multi-task learning of Gaussian graphical models. In Proceedings of the 27th International Conference on Machine Learning (ICML'10). 447--454. Google ScholarDigital Library
- John E. Hopcroft and Jeffrey D. Ullman. 1973. Set merging algorithms. SIAM J. Comput. 2, 4 (1973), 294–303.Google ScholarDigital Library
- Wuyungerile Li, Bing Jia, Haotian Xu, Zhaopeng Zong, and Takashi Watanabe. 2019. A multi-task scheduling mechanism based on ACO for maximizing workers’ benefits in mobile crowdsensing service markets with the internet of things. IEEE Access 7 (2019), 41463–41469.Google ScholarCross Ref
- Xiaoyuan Liu, Hongwei Li, Guowen Xu, Sen Liu, Zhe Liu, and Rongxing Lu. 2020. PADL: Privacy-aware and asynchronous deep learning for IoT applications. IEEE IoT J. (2020).Google Scholar
- Meng Ma, Ping Wang, and Chao-Hsien Chu. 2013. Data management for internet of things: Challenges, approaches and opportunities. In Proceedings of the 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing. IEEE, 1144–1151. Google ScholarDigital Library
- Rahul Mazumder and Trevor Hastie. 2012. Exact covariance thresholding into connected components for large-scale graphical lasso. J. Mach. Learn. Res. 13 (Mar. 2012), 781–794. Google ScholarDigital Library
- Tom M. Mitchell, Svetlana V. Shinkareva, Andrew Carlson, Kai-Min Chang, Vicente L. Malave, Robert A. Mason, and Marcel Adam Just. 2008. Predicting human brain activity associated with the meanings of nouns. Science 320, 5880 (2008), 1191–1195.Google Scholar
- Karthik Mohan, Maryam Fazel Palma London, Daniela Witten, and Su-In Lee. 2014. Node-based learning of multiple Gaussian graphical models. J. Mach. Learn. Res. 15, 1 (2014), 445. Google ScholarDigital Library
- Di Mu, Yunpeng Ge, Mo Sha, Steve Paul, Niranjan Ravichandra, and Souma Chowdhury. 2019. Robust optimal selection of radio type and transmission power for internet of things. ACM Trans. Sen. Netw. 15, 4, Article 39 (July 2019), 25 pages. DOI:https://doi.org/10.1145/3342516 Google ScholarDigital Library
- Michael A. Osborne, Stephen J. Roberts, Alex Rogers, and Nicholas R. Jennings. 2012. Real-time information processing of environmental sensor network data using bayesian gaussian processes. ACM Trans. Sen. Netw. 9, 1, Article 1 (Nov. 2012), 32 pages. DOI:https://doi.org/10.1145/2379799.2379800 Google ScholarDigital Library
- Rodrigo Perin, Martin Telefont, and Henry Markram. 2013. Computing the size and number of neuronal clusters in local circuits. Front. Neuroanat. 7 (2013), 1.Google ScholarCross Ref
- Russell A. Poldrack and Krzysztof J. Gorgolewski. 2017. OpenfMRI: Open sharing of task fMRI data. NeuroImage 144 (2017), 259–261.Google ScholarCross Ref
- Adam J. Rothman, Peter J. Bickel, Elizaveta Levina, Ji Zhu, et al. 2008. Sparse permutation invariant covariance estimation. Electr. J. Stat. 2 (2008), 494–515.Google ScholarCross Ref
- Adam J. Rothman, Elizaveta Levina, and Ji Zhu. 2009. Generalized thresholding of large covariance matrices. J. Am. Stat. Assoc. 104, 485 (2009), 177–186.Google ScholarCross Ref
- Mehrdad Salimitari, Shameek Bhattacharjee, Mainak Chatterjee, and Yaser P. Fallah. 2020. A prospect theoretic approach for trust management in IoT networks under manipulation attacks. ACM Trans. Sens. Netw. 16, 3, Article 26 (May 2020), 26 pages. DOI:https://doi.org/10.1145/3392058 Google ScholarDigital Library
- Paul T. Schultz and Robert A. Sartini. 2017. IoT communication utilizing secure asynchronous P2P communication and data exchange. US Patent 9,838,204.Google Scholar
- Shihao Shen, Yiwen Han, Xiaofei Wang, and Yan Wang. 2019. Computation offloading with multiple agents in edge-computing–supported IoT. ACM Trans. Sens. Netw. 16, 1, Article 8 (Dec. 2019), 27 pages. DOI:https://doi.org/10.1145/3372025 Google ScholarDigital Library
- Noah Simon, Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2013. A sparse-group lasso. J. Comput. Graph. Stat. 22, 2 (2013), 231–245.Google ScholarCross Ref
- Zhiyong Sun, Junyong Ye, TongqingWang, Shijian Huang, and Jin Luo. 2020. Behavioral feature recognition of multi-task compressed sensing with fusion relevance in the Internet of Things environment. Comput. Commun, 157 (2020), 381--393.Google ScholarCross Ref
- Erming Tian, Fenghuang Zhan, Ronald Walker, Erik Rasmussen, Yupo Ma, Bart Barlogie, and John D Shaughnessy Jr. 2003. The role of the Wnt-signaling antagonist DKK1 in the development of osteolytic lesions in multiple myeloma. N. Engl. J. Med. 349, 26 (2003), 2483–2494.Google ScholarCross Ref
- Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. Ser. B Methodol. 58, 1 (1996), 267–288.Google ScholarCross Ref
- Ivy F. Tso, Saige Rutherford, Yu Fang, Mike Angstadt, and Stephan F. Taylor. 2018. The “social brain” is highly sensitive to the mere presence of social information: An automated meta-analysis and an independent study. PLoS One 13, 5 (2018).Google Scholar
- Beilun Wang, Ji Gao, and Yanjun Qi. 2017. A fast and scalable joint estimator for learning multiple related sparse gaussian graphical models. arXiv:1702.02715. Retrieved from https://arxiv.org/abs/1702.02715.Google Scholar
- Beilun Wang, Ritambhara Singh, and Yanjun Qi. 2017. A constrained l1 minimization approach for estimating multiple sparse Gaussian or nonparanormal graphical models. Mach. Learn. 106, 9-10 (2017), 1381–1417. Google ScholarDigital Library
- Lizhe Wang and Rajiv Ranjan. 2015. Processing distributed internet of things data in clouds. IEEE Cloud Comput. 2, 1 (2015), 76–80.Google ScholarCross Ref
- Daniela M. Witten, Jerome H. Friedman, and Noah Simon. 2011. New insights and faster computations for the graphical lasso. J. Comput. Graph. Stat. 20, 4 (2011), 892–900.Google ScholarCross Ref
- Nuosi Wu, Jiang Huang, Xiao-Fei Zhang, Le Ou-Yang, Shan He, Zexuan Zhu, and Weixin Xie. 2019. Weighted fused pathway graphical lasso for joint estimation of multiple gene networks. Front. Genet. 10 (2019), 623.Google ScholarCross Ref
- Han Xiao, Changqiao Xu, Tengfei Cao, Lujie Zhong, and Gabriel-Miro Muntean. 2019. GTTC: A low-expenditure IoT multi-task coordinated distributed computing framework with fog computing. In Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM’19). IEEE, 1–6.Google ScholarDigital Library
- Eunho Yang, Aurélie C. Lozano, and Pradeep K. Ravikumar. 2014. Elementary estimators for graphical models. In Advances in Neural Information Processing Systems. 2159–2167. Google ScholarDigital Library
- Ming Yuan and Yi Lin. 2007. Model selection and estimation in the gaussian graphical model. Biometrika 94, 1 (2007), 19–35.Google ScholarCross Ref
- Bai Zhang and Yue Wang. 2012. Learning structural changes of Gaussian graphical models in controlled experiments. arXiv:1203.3532. Retrieved from https://arxiv.org/abs/1203.3532. Google ScholarDigital Library
- L. Zhang, S. Yu, X. Ding, and X. Wang. 2014. Research on IOT RESTful web service asynchronous composition based on BPEL. In Proceedings of the 2014 6th International Conference on Intelligent Human-Machine Systems and Cybernetics, Vol. 1. 62–65. Google ScholarDigital Library
- Yi Zhang and Jeff G. Schneider. 2010. Learning multiple tasks with a sparse matrix-normal penalty. In Advances in Neural Information Processing Systems. 2550–2558. Google ScholarDigital Library
- Yunzhang Zhu, Xiaotong Shen, and Wei Pan. 2014. Structural pursuit over multiple undirected graphs. J. Am. Stat. Assoc. 109, 508 (2014), 1683–1696.Google ScholarCross Ref
- Hui Zou. 2006. The adaptive lasso and its oracle properties. J. Am. Stat. Assoc. 101, 476 (2006), 1418–1429.Google ScholarCross Ref
Index Terms
- Scalable Estimator for Multi-task Gaussian Graphical Models Based in an IoT Network
Recommendations
IoT Mashups: From IoT Big Data to IoT Big Service
ICFNDS '17: Proceedings of the International Conference on Future Networks and Distributed SystemsInternet of Things (IoT) addresses the challenge to provide a transparent access to a huge number of IoT resources that can be either physical devices or just data resources. Moreover, because of the large number of resource-constrained devices and the ...
From IoT big data to IoT big services
SAC '17: Proceedings of the Symposium on Applied ComputingThe large-scale deployments of Internet of Things (IoT) systems have introduced several new challenges in terms of processing their data. The massive amount of IoT-generated data requires design solutions to speed up data processing, scale up with the ...
External integrity verification for outsourced big data in cloud and IoT
As cloud computing is being widely adopted for big data processing, data security is becoming one of the major concerns of data owners. Data integrity is an important factor in almost any data and computation related context. It is not only one of the ...
Comments