Skip to main content

Advertisement

SpringerLink
Log in

Deep Reinforcement Learning-based Resource Allocation for 5G Machine-type Communication in Active Distribution Networks with Time-varying Interference

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

Active distribution networks (ADNs) can solve the problem of grid compatibility and large-scale, intermittent, renewable energy applications. As the core part of ADNs, advanced metering infrastructure (AMI) meets the reliability requirements of the system for monitoring, diagnosis and control by extensive data acquisition and effective data transmission. The fifth-generation (5G) New Radio (NR) with ultra-reliable low-latency communication (URLLC) can be applied in ADNs for data transmission. However, in ADNs, the electromagnetic environment is complex, and the interference is diverse and time-varying. This scenario creates great challenges to data transmission in 5G communication networks. In this paper, we model the data transmission in 5G, design a rolling solution framework from predicting interference to improving data repetition, and then allocate wireless resources. To adapt resource allocation to time-varying interference, we propose an interference prediction algorithm to accurately estimate the interference distribution in the whole scheduling cycle. Moreover, to meet the second-level, resource scheduling requirement, we model resource allocation as a dynamic programming problem with the goal of maximizing energy efficiency and solve it by a DDQN-based reinforcement learning algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Neagu BC, Grigoraş G, Ivanov O (2019) The optimal operation of active distribution networks with smart systems. Advanced Communication and Control Methods for Future Smartgrids, pp 3

  2. Li Q, Tang H, Liu Z, Li J, Xu X, Sun W (2021) Optimal resource allocation of 5g machine-type communications for situation awareness in active distribution networks. IEEE Syst J, pp 1–11

  3. Vadari M (2019) The future of distribution operations and planning: the electric utility environment is changing. IEEE Power and Energy Magazine 18(1):18–25

    Article  Google Scholar 

  4. Kong P-Y, Song Y (2019) Joint consideration of communication network and power grid topology for communications in community smart grid. IEEE Transactions on Industrial Informatics 16(5):2895–2905

    Article  MathSciNet  Google Scholar 

  5. Al-Rubaye S, Al-Dulaimi A, Cosmas J (2016) Spectrum allocation techniques for industrial smart grid infrastructure. In: 2016 IEEE 14th International Conference on Industrial Informatics (INDIN), IEEE, pp 1036–1039

  6. Sachs J, Wikstrom G, Dudda T, Baldemair R, Kittichokechai K (2018) 5G radio network design for ultra-reliable low-latency communication. IEEE Network 32(2):24–31

    Article  Google Scholar 

  7. Pedersen KI, Khosravirad SR, Berardinelli G, Frederiksen F (2017) Rethink hybrid automatic repeat request design for 5g: Five configurable enhancements. IEEE Wirel Commun 24(6):154–160

    Article  Google Scholar 

  8. Rao J, Vrzic S (2018) Packet duplication for urllc in 5g: Architectural enhancements and performance analysis. IEEE Netw 32(2):32–40

    Article  Google Scholar 

  9. Aijaz A (2019) Packet duplication in dual connectivity enabled 5g wireless networks: overview and challenges. IEEE Communications Standards Magazine 3(3):20–28

    Article  Google Scholar 

  10. Mahmood NH, Abreu R, Böhnke R, Schubert M, Berardinelli G, Jacobsen TH (2019) Uplink grant-free access solutions for urllc services in 5g new radio. In: 2019 16th international symposium on wireless communication systems (ISWCS), IEEE, pp 607–612

  11. Anand A, De Veciana G, Shakkottai S (2020) Joint scheduling of urllc and embb traffic in 5g wireless networks. IEEE/ACM Trans Networking 28(2):477–490

    Article  Google Scholar 

  12. GT R1-1612246 (2016) Discussion on HARQ support for URLLC. In: RAN1 #87 Reno, Nevada

  13. Abreu R, Berardinelli G, Jacobsen T, Pedersen K, Mogensen P (2018) A blind retransmission scheme for ultra-reliable and low latency communications. In: 2018 IEEE 87th vehicular technology conference (VTC Spring), IEEE, pp 1–5

  14. Jacobsen T, Abreu R, Berardinelli G, Pedersen K, Kovács IZ, Mogensen P (2019) System level analysis of k-repetition for uplink grant-free urllc in 5g nr. In: European wireless 2019; 25th European wireless conference. VDE, pp 1–5

  15. Chang B, Zhang L, Li L, Zhao G, Chen Z (2019) Optimizing resource allocation in urllc for real-time wireless control systems. IEEE Trans Veh Technol 68(9):8916–8927

    Article  Google Scholar 

  16. Zhou Z, Ratasuk R, Mangalvedhe N, Ghosh A (2018) Resource allocation for uplink grant-free ultra-reliable and low latency communications. In: 2018 IEEE 87th vehicular technology conference (VTC Spring), IEEE, pp 1–5

  17. Zhao C, Cai Y, Liu A, Zhao M, Hanzo L (2020) Mobile edge computing meets mmwave communications: Joint beamforming and resource allocation for system delay minimization. IEEE Trans Wirel Commun 19(4):2382–2396

    Article  Google Scholar 

  18. Liu Z, Zhan C, Cui Y, Wu C, Hu H (2021) Robust edge computing in uav systems via scalable computing and cooperative computing. IEEE Wirel Commun 28(5):36–42

    Article  Google Scholar 

  19. Chen X, Wu C, Chen T, Liu Z, Zhang H, Bennis M, Liu H, Ji Y (2022) Information freshness-aware task offloading in air-ground integrated edge computing systems. IEEE Journal on Selected Areas in Communications 40(1):243–258

    Article  Google Scholar 

  20. Zhan C, Hu H, Liu Z, Wang Z, Mao S (2021) Multi-uav-enabled mobile-edge computing for time-constrained iot applications. IEEE Internet of Things Journal 8(20):15553–15567

    Article  Google Scholar 

  21. Li Q, Tang H, Sun W, Li W, Xu X (2020) An optimal wireless resource allocation of machine-type communications in the 5g network for situation awareness of active distribution network. In: 2020 IEEE international conference on communications, control, and computing technologies for smart grids (SmartGridComm), IEEE, pp 1–7

  22. Liu Z, Zhang C, Dong M, Gu B, Ji Y, Tanaka Y (2017) Markov-decision-process-assisted consumer scheduling in a networked smart grid. IEEE Access 5:2448–2458

    Article  Google Scholar 

  23. Yang T, Hu Y, Gursoy MC, Schmeink A, Mathar R (2018) Deep reinforcement learning based resource allocation in low latency edge computing networks. In: 2018 15th international symposium on wireless communication systems (ISWCS), IEEE, pp 1–5

  24. Liang Y, He Y, Zhong X (2020) Decentralized computation offloading and resource allocation in mec by deep reinforcement learning. In: 2020 IEEE/CIC international conference on communications in China (ICCC), IEEE, pp 244–249

  25. Li Q, Cheng H, Yang Y, Tang H, Liu Z, Cao Y, Sun W (2021) Deep reinforcement learning-based resource allocation for 5g machine-type communication in active distribution networks. In; International conference on mobile networks and management, Springer, pp 39–59

  26. Vinyals O, Fortunato M, Jaitly N (2015) Pointer networks. Advances in neural information processing systems, vol 28

  27. Mahmood NH, López OA, Alves H, Latva-Aho M (2020) A predictive interference management algorithm for urllc in beyond 5g networks. IEEE Commun Lett 25(3):995–999

    Article  Google Scholar 

  28. Albu MM, Sănduleac M, Stănescu C (2016) Syncretic use of smart meters for power quality monitoring in emerging networks. IEEE Trans Smart Grid 8(1):485–492

    Article  Google Scholar 

  29. Castellanos CU, Villa DL, Rosa C, Pedersen KI, Calabrese FD, Michaelsen P-H, Michel J (2008) Performance of uplink fractional power control in utran lte. In: VTC Spring 2008-IEEE vehicular technology conference, IEEE, pp 2517–2521

  30. Zhang Z, Zhang D, Qiu RC (2019) Deep reinforcement learning for power system applications: an overview. CSEE Journal of Power and Energy Systems 6(1):213–225

    Google Scholar 

  31. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533

    Article  Google Scholar 

  32. Van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with double q-learning. In: Proceedings of the AAAI conference on artificial intelligence, vol 30, no 1

Download references

Acknowledgments

This work is supported in part by grants from the National Natural Science Foundation of China (52077049, 51877060, 62173120), the Anhui Provincial Natural Science Foundation (2008085UD04), and the 111 Project (BP0719039).

Author information

Authors and Affiliations

  1. School of Electrical Engineering and Automation, Hefei University of Technology, Hefei, 230041, Anhui, China

    Qiyue Li, Hong Cheng, Haochen Tang, Guojun Luo & Wei Sun

  2. Department, Engineering Technology Research Center of Industrial Automation, Hefei, 230041, Anhui, China

    Qiyue Li, Hong Cheng, Haochen Tang, Guojun Luo & Wei Sun

  3. Shenzhen Cyberaray Network Technology Co., Ltd, Shenzhen, China

    Yangzhao Yang

  4. Guangdong Provincial Key Laboratory of Intelligent Transportation Sys-tem, School of Intelligent Systems Engineering, Sun Yat-sen University, Guangdong, China

    Junbo Wang

Authors
  1. Qiyue Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hong Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yangzhao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haochen Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Junbo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guojun Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Wei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Sun.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Q., Cheng, H., Yang, Y. et al. Deep Reinforcement Learning-based Resource Allocation for 5G Machine-type Communication in Active Distribution Networks with Time-varying Interference. Mobile Netw Appl 27, 2264–2279 (2022). https://doi.org/10.1007/s11036-022-02006-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-022-02006-5

Keywords

Search

Navigation