In order to meet the dramatic wireless bandwidth demands of emerging multimedia applications, cognitive radio has been proposed as one of promising solutions to improve the spectrum efficiency. This article aims at pursuing high spectrum efficiency via accessing the idle spectrum intelligently without information exchange among users. Different from infrastructure-based wireless networks, users in cognitive radio networks tend to compete with each other to access limited idle spectrum, thus leading to a dynamically heterogeneous radio environment. In this article, a Q-learning based spectrum access scheme is proposed to adaptively allocate multimedia data over multiple idle spectrum holes. Taking into consideration the rigorous delay and throughput performance requirements of multimedia applications, we integrate these two indicators into the definition of reward function in the proposed Q-learning algorithm. The simulation results show that the proposed scheme can quickly converge to a stable state in terms of throughput, power efficiency, and collision probability. Furthermore, the proposed learning rate adjustment strategy makes the performance of the spectrum access algorithm converge the quickest and only consumes 78% time to achieve the targeted collision probability, i.e., 0.1, compared with two other typical parameter adjustment strategies.

中文翻译：

---

基于Q学习的频谱访问在认知无线电网络上的多媒体传输

为了满足新兴多媒体应用对无线带宽的巨大需求，已经提出了认知无线电作为提高频谱效率的有前途的解决方案之一。本文旨在通过智能访问空闲频谱来实现高频谱效率，而无需用户之间进行信息交换。与基于基础架构的无线网络不同，认知无线电网络中的用户倾向于相互竞争以访问有限的空闲频谱，从而导致动态异构的无线电环境。在本文中，提出了一种基于Q学习的频谱访问方案，以在多个空闲频谱孔上自适应地分配多媒体数据。考虑到多媒体应用程序严格的延迟和吞吐量性能要求，我们在建议的Q学习算法中将这两个指标整合到奖励函数的定义中。仿真结果表明，该方案可以在吞吐量，功率效率和碰撞概率方面迅速收敛到稳定状态。此外，与另外两种典型的参数调整策略相比，所提出的学习速率调整策略使频谱访问算法的性能收敛最快，并且仅花费78％的时间来达到目标​​碰撞概率，即0.1。