This paper describes the theoretical principles and experimental results of reinforcement learning algorithms embedded into IoT devices (Internet of Things), in order to tackle the problem of radio collision mitigation in ISM unlicensed bands. Multi-armed bandit (MAB) learning algorithms are used here to improve both the IoT network capability to support the expected massive number of objects and the energetic autonomy of the IoT devices. We first illustrate the efficiency of the proposed approach in a proof-of-concept, based on USRP software radio platforms operating on real radio signals. It shows how collisions with other RF signals are diminished for IoT devices that use MAB learning. Then we describe the first implementation of such algorithms on LoRa devices operating in a real LoRaWAN network at 868 MHz. We named this solution IoTligent. IoTligent does not add neither processing overhead, so it can be run into the IoT devices, nor network overhead, so that it requires no change to LoRaWAN protocol. Real-life experiments done in a real LoRa network show that IoTligent devices’ battery life can be extended by a factor of 2, in the scenarios we faced during our experiment. Finally we submit IoTligent devices to very constrained conditions that are expected in the future with the growing number of IoT devices, by generating an artificial IoT massive radio traffic in anechoic chamber. We show that IoTligent devices can cope with spectrum scarcity that will occur at that time in unlicensed bands.

本文介绍了嵌入在IoT设备（物联网）中的强化学习算法的理论原理和实验结果，以解决ISM非授权频段中的无线电冲突缓解问题。此处使用多臂匪（MAB）学习算法来改善IoT网络功能以支持预期的大量对象，并提高IoT设备的能量自主性。我们首先在基于实际无线电信号的USRP软件无线电平台的概念验证中说明所提出方法的效率。它显示了如何使用MAB学习的IoT设备减少与其他RF信号的冲突。然后，我们描述了在真正的LoRaWAN网络中以868 MHz运行的LoRa设备上此类算法的首次实现。我们将此解决方案命名为IoTligent。IoTligent既不会增加处理开销，因此可以在IoT设备中运行，也不会增加网络开销，因此无需更改LoRaWAN协议。在真实的LoRa网络中进行的实际实验表明，在我们在实验过程中遇到的场景中，IoTligent设备的电池寿命可以延长2倍。最后，我们通过在消声室中生成人工IoT大规模无线电流量，使IoTligent设备处于非常受限的条件下，这在未来随着IoT设备数量的增加而有望达到。我们表明，IoTligent设备可以应对当时在无执照频段中出现的频谱短缺问题。在真实的LoRa网络中进行的现实生活实验表明，在我们在实验过程中遇到的场景中，IoTligent设备的电池寿命可以延长2倍。最后，我们通过在消声室中生成人工的IoT大规模无线电流量，使IoTligent设备处于非常受限的条件下，随着IoT设备数量的增加，这种情况将在未来出现。我们表明，IoTligent设备可以应对当时在无执照频段中出现的频谱短缺问题。在真实的LoRa网络中进行的实际实验表明，在我们在实验过程中遇到的场景中，IoTligent设备的电池寿命可以延长2倍。最后，我们通过在消声室中生成人工的IoT大规模无线电流量，使IoTligent设备处于非常受限的条件下，随着IoT设备数量的增加，这种情况将在未来出现。我们表明，IoTligent设备可以应对当时在无执照频段中出现的频谱短缺问题。