Computation offloading is a promising solution for resource-limited IoT devices to accomplish computation-intensive tasks. In order to promote the service trading between edge computing service providers and IoT devices, a series of works have explored incentive mechanisms for IoT-edge computing. However, most traditional incentive mechanisms (such as stackelberg game-based approaches) expose privacy of participants. Moreover, the existing reinforcement learning-based incentive mechanisms do not consider the competition among multiple providers, which is not in line with reality. In this paper, taking privacy concern and competition among providers into consideration, we utilize reinforcement learning (RL) technique to design a privacy-preserving incentive mechanism for multiple providers and multiple IoT devices. Specifically, we model the pricing and demand problem of providers and IoT devices as a multi-leader multi-follower stackelberg game, in which the providers work as leaders to determine their prices first, and then the IoT devices determine their demands as followers. We prove the existence and uniqueness of the Nash equilibrium (NE) of this game. Due to privacy concern, providers and IoT devices are unwilling to disclose their own parameters, which makes the derivation of NE becoming a great challenge. To address this problem, a new RL-based pricing mechanism (RLPM) is proposed, which enables providers to learn their optimal pricing strategies without knowing private information of other participants. Finally, numerical simulations are conducted to illustrate the convergence and effectiveness of the RLPM compared with other existing algorithms.

对于资源有限的IoT设备，计算分流是一种有前途的解决方案，可以完成计算密集型任务。为了促进边缘计算服务提供商与物联网设备之间的服务交易，一系列工作探索了物联网边缘计算的激励机制。但是，大多数传统的激励机制（例如基于Stackelberg游戏的方法）都暴露了参与者的隐私。而且，现有的基于强化学习的激励机制没有考虑多个提供商之间的竞争，这与现实不符。在本文中，考虑到隐私问题以及提供商之间的竞争，我们利用强化学习（RL）技术设计了针对多个提供商和多个IoT设备的隐私保护激励机制。特别，我们将供应商和物联网设备的定价和需求问题建模为一个多领导者，多从业者的Stackelberg游戏，在该博弈中，供应商作为领导者首先确定其价格，然后物联网设备将其需求确定为跟随者。我们证明了该游戏的纳什均衡（NE）的存在和唯一性。由于隐私问题，提供商和物联网设备不愿透露自己的参数，这使得网元的推导成为一个巨大的挑战。为了解决这个问题，提出了一种新的基于RL的定价机制（RLPM），该机制使提供商能够了解其最优定价策略，而无需了解其他参与者的私人信息。最后，进行了数值模拟，以说明与其他现有算法相比，RLPM的收敛性和有效性。