Offloading cellular traffic via Device-to-Device communication (or D2D offloading) has been proved to be an effective way to ease the traffic burden of cellular networks. However, mobile nodes may not be willing to take part in D2D offloading without proper financial incentives since the data offloading process will incur a lot of resource consumption. Therefore, it is imminent to exploit effective incentive mechanisms to motivate nodes to participate in D2D offloading. Furthermore, the design of the content caching strategy is also crucial to the performance of D2D offloading. In this paper, considering these issues, a novel Incentive-driven and Deep Q Network (DQN) based Method, named IDQNM is proposed, in which the reverse auction is employed as the incentive mechanism. Then, the incentive-driven D2D offloading and content caching process is modeled as Integer Non-Linear Programming (INLP), aiming to maximize the saving cost of the Content Service Provider (CSP). To solve the optimization problem, the content caching method based on a Deep Reinforcement Learning (DRL) algorithm, named DQN is proposed to get the approximate optimal solution, and a standard Vickrey-Clarke-Groves (VCG)-based payment rule is proposed to compensate for mobile nodes' cost. Extensive real trace-driven simulation results demonstrate that the proposed IDQNM greatly outperforms other baseline methods in terms of the CSP's saving cost and the offloading rate in different scenarios.

中文翻译：

---

用于内容缓存和 D2D 卸载的激励驱动深度强化学习


通过设备到设备通信（或 D2D 卸载）卸载蜂窝流量已被证明是减轻蜂窝网络流量负担的有效方法。然而，如果没有适当的经济激励，移动节点可能不愿意参与D2D卸载，因为数据卸载过程会产生大量的资源消耗。因此，利用有效的激励机制来激励节点参与D2D卸载迫在眉睫。此外，内容缓存策略的设计对于D2D卸载的性能也至关重要。考虑到这些问题，本文提出了一种基于激励驱动和深度Q网络（DQN）的新方法，称为IDQNM，其中采用反向拍卖作为激励机制。然后，将激励驱动的D2D卸载和内容缓存过程建模为整数非线性规划（INLP），旨在最大限度地节省内容服务提供商（CSP）的成本。为了解决优化问题，提出了基于深度强化学习（DRL）算法的内容缓存方法（称为DQN）以获得近似最优解，并提出了基于标准Vickrey-Clarke-Groves（VCG）的支付规则补偿移动节点的成本。大量真实跟踪驱动的仿真结果表明，所提出的 IDQNM 在不同场景下的 CSP 节省成本和卸载率方面远远优于其他基线方法。