Device-to-device (D2D) communication is a promising paradigm to meet the requirement of ultra-dense, low-latency and high-rate in the fifth-generation networks. However, energy consumption is a critical issue for the D2D communication, especially for D2D relay networks. To make the best use of D2D communication, the problem of optimizing energy efficiency (EE) must be addressed. In this paper, we propose a joint power allocation and relay selection (JPARS) scheme for the improvement of energy efficiency in relay-aided D2D communications underlaying cellular network. A mixed integer nonlinear fractional programming (MINLP) problem of the total EE for D2D pairs is formulated. While ensuring the quality of service (QoS) of cellular users and D2D links, we solve the power allocation problem by Dinkelbach method and Lagrange dual decomposition. After that, Q-learning, one of the reinforcement learning algorithms, is employed to solve the relay selection problem. Finally, we provide in-depth theoretical analysis of the proposed scheme in terms of complexity and signaling overhead. Simulation results verify that the proposed scheme not only overcomes the bottleneck effect, but also nearly reaches the theoretical maximum in terms of the total EE of D2D pairs.

中文翻译：

---

用于中继辅助 D2D 通信的节能功率分配和基于 Q 学习的中继选择

设备到设备（D2D）通信是满足第五代网络中超密集、低延迟和高速率要求的一种很有前景的范式。然而，能耗是 D2D 通信的一个关键问题，尤其是对于 D2D 中继网络。为了充分利用 D2D 通信，必须解决优化能源效率 (EE) 的问题。在本文中，我们提出了一种联合功率分配和中继选择 (JPARS) 方案，以提高中继辅助 D2D 通信底层蜂窝网络的能源效率。制定了 D2D 对总 EE 的混合整数非线性分数规划 (MINLP) 问题。在保证蜂窝用户和 D2D 链路的服务质量 (QoS) 的同时，我们通过 Dinkelbach 方法和拉格朗日对偶分解解决了功率分配问题。之后，采用强化学习算法之一的 Q-learning 来解决中继选择问题。最后，我们在复杂性和信令开销方面对所提出的方案进行了深入的理论分析。仿真结果验证了所提出的方案不仅克服了瓶颈效应，而且在 D2D 对的总 EE 方面几乎达到了理论最大值。