The 5G heterogeneous network architecture integrates different radio access technologies (RATs), which will support the large-scale communication connection of massive Internet-of-Things (IoT) devices. However, as the rapid growth of IoT connections, personalized requirements of services requested and heterogeneity deepening of the network system, how to design an intelligent network selection scheme for user devices (UDs) is becoming a crucial challenge in the 5G heterogeneous network system. Most of the existing network selection methods only optimize the selection strategies from the user side or network side, which results in heavy network congestion, poor user experience, and system performance degradation. Accordingly, we propose a multiagent $Q$ -learning network selection (MAQNS) algorithm based on Nash $Q$ -learning, which can learn a joint optimal selection strategy to improve system throughput and reduce user blocking on the premise of ensuring the requirements of IoT services. In particular, we apply the discrete-time Markov chains to model the network selection, and the analytic hierarchy process (AHP) and gray relation analysis (GRA) are jointly utilized to obtain user preferences for each network. Finally, performance evaluation demonstrates that comparing to the existing schemes, MAQNS proposed cannot only improve system throughput and reduce user blocking but also promote user experience on average energy efficiency and delay.

中文翻译：

---

5G异构网络系统中多业务用户智能选网算法：Nash Q-Learning方法

5G异构网络架构集成了不同的无线接入技术（RAT），将支持海量物联网（IoT）设备的大规模通信连接。然而，随着物联网连接的快速增长、服务请求的个性化需求以及网络系统异构性的深化，如何设计智能的用户设备（UDs）选网方案成为5G异构网络系统中的关键挑战。现有的网络选择方法大多只从用户侧或网络侧优化选择策略，导致网络拥塞严重、用户体验差、系统性能下降。因此，我们提出了一个多代理 $Q$ -基于Nash的学习网络选择（MAQNS）算法 $Q$ -learning，可以在保证物联网服务需求的前提下，学习联合最优选择策略，提高系统吞吐量，减少用户阻塞。特别是，我们应用离散时间马尔可夫链对网络选择进行建模，并联合使用层次分析法（AHP）和灰色关系分析（GRA）来获取每个网络的用户偏好。最后，性能评估表明，与现有方案相比，所提出的 MAQNS 不仅提高了系统吞吐量，减少了用户阻塞，而且在平均能效和延迟方面提升了用户体验。