The emerging technology of reconfigurable intelligent surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this article, we consider multiuser and multi-RIS-empowered wireless systems and present a thorough survey of the online machine learning approaches for the orchestration of their various tunable components. Focusing on the sum-rate maximization as a representative design objective, we present a comprehensive problem formulation based on deep reinforcement learning (DRL). We detail the correspondences among the parameters of the wireless system and the DRL terminology, and devise generic algorithmic steps for the artificial neural network training and deployment while discussing their implementation details. Further practical considerations for multi-RIS-empowered wireless communications in the sixth-generation (6G) era are presented along with some key open research challenges. Different from the DRL-based status quo, we leverage the independence between the configuration of the system design parameters and the future states of the wireless environment, and present efficient multiarmed bandits approaches, whose resulting sum-rate performances are numerically shown to outperform random configurations, while being sufficiently close to the conventional deep QQ network (DQN) algorithm, but with lower implementation complexity.

中文翻译：

---

可重构智能表面支持智能无线电环境的普遍机器学习


新兴的可重构智能表面 (RIS) 技术是智能无线环境的推动者，它提供了高度可扩展、低成本、硬件高效且几乎能源中性的解决方案，用于动态控制电磁信号在无线网络中的传播。无线介质，最终为不同的操作目标提供增强的环境智能。在这种可重新配置的无线电环境中设想密集部署 RIS 的主要挑战之一是在计算硬件有限甚至不存在的情况下高效配置多个超表面。在本文中，我们考虑多用户和多 RIS 授权的无线系统，并对用于编排各种可调组件的在线机器学习方法进行全面调查。我们将总和率最大化作为代表性设计目标，提出了基于深度强化学习（DRL）的综合问题表述。我们详细介绍了无线系统参数和 DRL 术语之间的对应关系，并为人工神经网络训练和部署设计通用算法步骤，同时讨论其实现细节。提出了第六代 (6G) 时代多 RIS 授权无线通信的进一步实际考虑因素以及一些关键的开放研究挑战。 与基于 DRL 的现状不同，我们利用系统设计参数的配置与无线环境的未来状态之间的独立性，提出了高效的多臂老虎机方法，其所得的总速率性能在数值上显示优于随机配置，同时足够接近传统的深度QQ网络（DQN）算法，但实现复杂度较低。