Energy-efficient communication remains one of the key requirements of the Internet of Things (IoT) platforms. The concern on energy consumption can be mitigated by exploiting technical ploys to reduce the volume of data for transmission (e.g., via sensing data compression) as well as by resorting to technological advancements (e.g., energy harvesting). However, these mitigating measures carry their own cost, which is the additional complexity of control and optimization in the digital communication chain. In particular, compression ratio is another control knob that needs adjusting besides the usual transmission parameters. Also, with the random and sporadic nature of the harvested energy, the goal shifts from mere energy conservation to judicious consumption of the renewable energy in a foresighted manner. In this paper, we assume an energy-harvesting IoT device that is tasked with (loss-lessly) compressing and reporting delay-constrained sensing events to an IoT gateway over a time-varying wireless channel. We are interested in computing an optimal policy for joint compression and transmission control adaptive to the node’s energy availability, transmission buffer length, as well as its wireless channel conditions. We cast the problem as a Constrained Markov Decision Process (CMDP), and propose a two-timescale model-free reinforcement learning (RL) algorithm that is able to shape the optimal control policy in the absence of the statistical knowledge of the underlying system dynamics. Exhaustive simulation experiments are conducted to investigate the convergence of the learning algorithm, to explore the impacts of different system parameters (such as: the rate of sensing events, the energy arrival rate, and battery capacity) on the performance of the proposed policy, as well as to compare against some baseline schemes.

节能通信仍然是物联网（IoT）平台的关键要求之一。通过利用技术手段减少传输数据量（例如，通过感测数据压缩）以及借助技术进步（例如，能量收集），可以减轻对能源消耗的担忧。但是，这些缓解措施要自己承担成本，这是数字通信链中控制和优化的额外复杂性。特别地，压缩比是除了通常的变速器参数之外还需要调节的另一个控制旋钮。另外，由于所收集的能量具有随机性和零星性，因此目标从单纯的节能转变为以可预见的方式明智地消耗可再生能源。在本文中，我们假设一个能量收集IoT设备的任务是（无损地）压缩并通过时变无线信道将延迟受限的传感事件报告给IoT网关。我们感兴趣的是，针对节点的能量可用性，传输缓冲区长度及其无线信道条件，为联合压缩和传输控制计算最佳策略。我们将此问题作为约束马尔可夫决策过程（CMDP）进行了建模，并提出了一种两时尺度的无模型强化学习（RL）算法，该算法能够在没有底层系统动力学的统计知识的情况下，形成最佳控制策略。 。进行了详尽的仿真实验以研究学习算法的收敛性，以探索不同系统参数（例如：