In this paper, we study an intelligent reflecting surface (IRS)-aided wireless secure communication system, where an IRS is deployed to adjust its reflecting elements to secure the communication of multiple legitimate users in the presence of multiple eavesdroppers. Aiming to improve the system secrecy rate, a design problem for jointly optimizing the base station (BS)'s beamforming and the IRS's reflecting beamforming is formulated considering different quality of service (QoS) requirements and time-varying channel conditions. As the system is highly dynamic and complex, and it is challenging to address the non-convex optimization problem, a novel deep reinforcement learning (DRL)-based secure beamforming approach is firstly proposed to achieve the optimal beamforming policy against eavesdroppers in dynamic environments. Furthermore, post-decision state (PDS) and prioritized experience replay (PER) schemes are utilized to enhance the learning efficiency and secrecy performance. Specifically, a modified PDS scheme is presented to trace the channel dynamic and adjust the beamforming policy against channel uncertainty accordingly. Simulation results demonstrate that the proposed deep PDS-PER learning based secure beamforming approach can significantly improve the system secrecy rate and QoS satisfaction probability in IRS-aided secure communication systems.

中文翻译：

---

基于深度强化学习的智能反射表面，用于安全无线通信


在本文中，我们研究了一种智能反射面（IRS）辅助的无线安全通信系统，其中部署IRS来调整其反射元件，以在存在多个窃听者的情况下保护多个合法用户的通信。为了提高系统保密性，考虑不同服务质量(QoS)要求和时变信道条件，提出了联合优化基站(BS)波束成形和IRS反射波束成形的设计问题。由于系统高度动态且复杂，解决非凸优化问题具有挑战性，因此首先提出了一种基于深度强化学习（DRL）的安全波束形成方法，以实现动态环境中针对窃听者的最优波束形成策略。此外，利用决策后状态（PDS）和优先经验重放（PER）方案来提高学习效率和保密性能。具体来说，提出了一种改进的PDS方案来跟踪信道动态并相应地针对信道不确定性调整波束形成策略。仿真结果表明，所提出的基于深度PDS-PER学习的安全波束成形方法可以显着提高IRS辅助安全通信系统中的系统保密率和QoS满足概率。