The explosive growth of machine-type communications (MTC) devices poses critical challenges to the existing cellular networks. Therefore, how to support massive MTC devices with limited resources is an urgent problem to be solved. Bursty traffic is an important characteristic of MTC devices, which makes it difficult for agents to learn useful experience and has a negative impact on model convergence. However, most existing reinforcement learning-based literatures assume that devices have saturate data. Towards this end, we propose two distributed Q-learning aided uplink grant-free non-orthogonal multiple access (NOMA) schemes (including all-devices distributed Q-learning (ADDQ) scheme and portion-devices distributed Q-learning (PDDQ) scheme) to maximize the number of accessible devices, where the bursty traffic of massive MTC devices is carefully considered. In order to reduce the dimension of scheduling space and mitigate the impact of bursty traffic, the idea of grouping devices as well as transmission resources and the intermittent learning mode are adopted in our schemes. Extensive numerical results demonstrate the advantages of proposed schemes from multiple perspectives.

中文翻译：

---

用于大规模机器类型通信的分布式 Q-Learning 辅助上行链路无赠款 NOMA


机器类型通信（MTC）设备的爆炸式增长对现有蜂窝网络提出了严峻的挑战。因此，如何用有限的资源支持海量的MTC设备是一个亟待解决的问题。突发流量是MTC设备的一个重要特征，这使得智能体难以学习有用的经验，并对模型收敛产生负面影响。然而，大多数现有的基于强化学习的文献都假设设备具有饱和数据。为此，我们提出了两种分布式Q学习辅助上行链路无授权非正交多址（NOMA）方案（包括所有设备分布式Q学习（ADDQ）方案和部分设备分布式Q学习（PDDQ）方案）以最大化可访问设备的数量，其中仔细考虑了海量MTC设备的突发流量。为了减少调度空间的维数并减轻突发流量的影响，我们的方案采用了设备和传输资源分组的思想以及间歇学习模式。大量的数值结果从多个角度证明了所提出方案的优势。