Skip to main content
SpringerLink
Log in

Determining node duty cycle using Q-learning and linear regression for WSN

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Wireless sensor network (WSN) is effective for monitoring the target environment, which consists of a large number of sensor nodes of limited energy. An efficient medium access control (MAC) protocol is thus imperative to maximize the energy efficiency and performance of WSN. The most existing MAC protocols are based on the scheduling of sleep and active period of the nodes, and do not consider the relationship between the load condition and performance. In this paper a novel scheme is proposed to properly determine the duty cycle of the WSN nodes according to the load, which employs the Q-learning technique and function approximation with linear regression. This allows low-latency energy-efficient scheduling for a wide range of traffic conditions, and effectively overcomes the limitation of Q-learning with the problem of continuous state-action space. NS3 simulation reveals that the proposed scheme significantly improves the throughput, latency, and energy efficiency compared to the existing fully active scheme and S-MAC.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alemdar A, Ibnkahla M. Wireless sensor networks: applications and challenges. In: Proceedings of the 9th International Symposium on Signal Processing and Its Applications. 2007, 1–6

  2. AlSkaif T, Bellalta B, Zapata M G, Ordinas J M B. Energy efficiency of MAC protocols in low data rate wireless multimedia sensor networks: a comparative study. Ad Hoc Networks, 2017, 56: 141–157

    Article  Google Scholar 

  3. Van Dam T, Langendoen K. An adaptive energy-efficient MAC protocol for wireless sensor networks. In: Proceedings of the 1st International Conference on Embedded Networked Sensor Systems. 2003, 171–180

  4. Zheng R, Kravets R. On-demand power management for ad hoc networks. Ad Hoc Networks, 2005, 3(1): 51–68

    Article  Google Scholar 

  5. Zheng R, Hou J C, Sha L. Asynchronous wakeup for ad hoc networks. In: Proceedings of the 4th ACM International Symposium on Mobile Ad Hoc Networking & Computing. 2003, 35–45

  6. Ye W, Heidemann J, Estrin D. An energy-efficient MAC protocol for wireless sensor networks. In: Proceedings of the 21st Annual Joint Conference of the IEEE Computer and Communications Societies. 2002, 1567–1576

  7. Ye W, Heidemann J, Estrin D. Medium access control with coordinated adaptive sleeping for wireless sensor networks. IEEE/ACM Transactions on Networking (ToN), 2004, 12(3): 493–506

    Article  Google Scholar 

  8. Jung E S, Vaidya N H. An energy efficient MAC protocol for wireless LANs. In: Proceedings of the 21st Annual Joint Conference of the IEEE Computer and Communications Societies. 2002, 1756–1764

  9. Liu S, Fan K W, Sinha P. CMAC: an energy-efficient MAC layer protocol using convergent packet forwarding for wireless sensor networks. ACM Transactions on Sensor Networks (TOSN), 2009, 5(4): 29

    Article  Google Scholar 

  10. Lewis F L, Vamvoudakis K G. Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2011, 41(1): 14–25

    Article  Google Scholar 

  11. Kosunalp S, Chu Y, Mitchell P D, Grace D, Clarke T. Use of Q-learning approaches for practical medium access control in wireless sensor networks. Engineering Applications of Artificial Intelligence, 2016, 55: 146–154

    Article  Google Scholar 

  12. Polastre J, Hill J, Culler D. Versatile low power media access for wireless sensor networks. In: Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems. 2004, 95–107

  13. Du S, Saha A K, Johnson D B. RMAC: a routing-enhanced duty-cycle MAC protocol for wireless sensor networks. In: Proceedings of IEEE INFOCOM 2007 — the 26th IEEE International Conference on Computer Communications. 2007, 1478–1486

  14. Tong F, Tang W, Xie R, Shu L, Kim Y C.P-MAC: a cross-layer duty cycle MAC protocol towards pipelining for wireless sensor networks. In: Proceedings of IEEE International Conference on Communications (ICC). 2011, 1–5

  15. Lin P, Qiao C, Wang X. Medium access control with a dynamic duty cycle for sensor networks. In: Proceedings of IEEE Wireless Communications and Networking Conference. 2004, 1534–1539

  16. Polastre J, Hill J, Culler D. Versatile low power media access for wireless sensor networks. In: Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems. 2004, 95–107

  17. Buettner M, Yee G V, Anderson E, Han R. X-MAC: a short preamble MAC protocol for duty-cycled wireless sensor networks. In: Proceedings of the 4th International Conference on Embedded Networked Sensor Systems. 2006, 307–320

  18. Sun Y, Gurewitz O, Johnson D B. RI-MAC: a receiver-initiated asynchronous duty cycle MAC protocol for dynamic traffic loads in wireless sensor networks. In: Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems. 2008, 1–14

  19. Niu J, Deng Z. Distributed self-learning scheduling approach for wireless sensor network. Ad Hoc Networks, 2013, 11(4): 1276–1286

    Article  Google Scholar 

  20. Sutton R S, Barto A G. Reinforcement Learning: An Introduction. MIT Press, 2018

Download references

Acknowledgements

This work was partly supported by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIT) (No. 2016-0-00133, Research on Edge computing via collective intelligence of hyper-connection IoT nodes), Korea, under the National Program for Excellence in SW supervised by the IITP (Institute for Information & communications Technology Promotion) (2015-0-00914), Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (2016R1A6A3A11931385, Research of key technologies based on software defined wireless sensor network for real time public safety service, 2017R1A2B2009095, Research on SDN-based WSN Supporting Realtime Stream Data Processing and Multi-connectivity), the second Brain Korea 21 PLUS project.

Author information

Authors and Affiliations

  1. College of Software, Sungkyunkwan University, Suwon, 440-746, Korea

    Han Yao Huang, Kyung Tae Kim & Hee Yong Youn

Authors
  1. Han Yao Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kyung Tae Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hee Yong Youn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hee Yong Youn.

Additional information

Han Yao Huang is currently a PhD student in College of Information and Communication Engineering from Sungkyunkwan University, Korea. He received BS degree from the University of Electronic Science and Technology of China, China in 2015. His current research interests include wireless networks, internet of things technology, and machine learning.

Kyung Tae Kim received the PhD degree in College of Information and Communication Engineering from Sungkyunkwan University, Korea in 2013. He is currently a research professor at the college of software from Sungkyunkwan University, Korea. His current research interests include ubiquitous computing, wireless networks, and internet of things technology.

Hee Yong Youn received the BS and MS in electrical engineering from Seoul National University, Korea in 1977 and 1979, respectively, and the PhD in computer engineering from the University of Massachusetts at Amherst, USA in 1988. He had been Associate Professor of Department of Computer Science and Engineering, The University of Texas at Arlington until 1999. He is Professor of College of Information and Communication Engineering and Director of Ubiquitous Computing Technology Research Institute, Sungkyunkwan University, Korea, and he is presently visiting SW R&D Center, Device Solutions, and Samsung Electronics. His research interests include cloud and ubiquitous computing, system software and middleware, and RFID/USN. He has published numerous papers and received Outstanding Paper Award from the 1988 IEEE International Conference on Distributed Computing Systems, 1992 Supercomputing, IEEE 2012 Int’l Conference on Computer, Information and Telecommunication Systems, and CyberC 2014. Prof. Youn has been General Co-Chair of IEEE PRDC 2001, Int’l Conf. on Ubiquitous Computing Systems (UCS) in 2006 and 2009, UbiComp 2008, CyberC 2010, Program Chair of PDCS 2003 and UCS 2007, respectively.

Electronic Supplementary Material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, H.Y., Kim, K.T. & Youn, H.Y. Determining node duty cycle using Q-learning and linear regression for WSN. Front. Comput. Sci. 15, 151101 (2021). https://doi.org/10.1007/s11704-020-9153-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-020-9153-6

Keywords

Search

Navigation