Energy management and power allocation policy is considered for energy harvesting (EH) communications. In this letter, we propose a joint optimization problem with the continuous EH time and transmit power to maximize the long-term throughput based on deep deterministic policy gradient (DDPG). However, the joint optimization problem leads to a large continuous action space. In order to reduce the dimension of action space, we present a deep reinforcement learning (DRL) framework by combining DDPG and convex program. The original problem is decomposed into two-layer optimization subproblems by using the primal decomposition method. The primary problem can be solved by DDPG with a low-dimensional action space. The lower-layer subproblem can be solved by using the existing convex toolbox. Numerical simulation results show that, compared with the existing energy management or power allocation policies for EH communications, the proposed DRL framework can achieve higher long-term throughput.

中文翻译：

---

基于DDPG的EH通信联合EH时间和发射功率优化

能量管理和功率分配策略被考虑用于能量收集 (EH) 通信。在这封信中，我们提出了一个具有连续 EH 时间和发射功率的联合优化问题，以基于深度确定性策略梯度（DDPG）最大化长期吞吐量。然而，联合优化问题会导致一个很大的连续动作空间。为了减少动作空间的维度，我们通过结合 DDPG 和凸程序提出了一个深度强化学习 (DRL) 框架。利用原始分解方法将原问题分解为两层优化子问题。主要问题可以通过具有低维动作空间的 DDPG 来解决。下层子问题可以通过使用现有的凸工具箱来解决。数值模拟结果表明，