In this paper, an IRS-aided integrated sensing and communications (ISAC) system operating in the terahertz (THz) band is proposed to maximize the system capacity. Transmit beamforming and phase-shift design are transformed into a universal optimization problem with ergodic constraints. Then the joint optimization of transmit beamforming and phase-shift design is achieved by gradient-based, primal-dual proximal policy optimization (PPO) in the multi-user multiple-input single-output (MISO) scenario. Specifically, the actor part generates continuous transmit beamforming and the critic part takes charge of discrete phase shift design. Based on the MISO scenario, we investigate a distributed PPO (DPPO) framework with the concept of multi-threading learning in the multi-user multiple-input multiple-output (MIMO) scenario. Simulation results demonstrate the effectiveness of the primal-dual PPO algorithm and its multi-threading version in terms of transmit beamforming and phase-shift design.

中文翻译：

---

太赫兹频段 IRS 辅助 ISAC 系统中基于近端策略优化的发射波束成形和相移设计


在本文中，提出了一种在太赫兹（THz）频段运行的IRS辅助集成传感和通信（ISAC）系统，以最大限度地提高系统容量。发射波束形成和相移设计转化为具有遍历约束的通用优化问题。然后，在多用户多输入单输出（MISO）场景中，通过基于梯度的原对偶近端策略优化（PPO）实现发射波束成形和相移设计的联合优化。具体来说，执行者部分生成连续的发射波束形成，而批评者部分负责离散相移设计。基于MISO场景，我们研究了多用户多输入多输出（MIMO）场景中具有多线程学习概念的分布式PPO（DPPO）框架。仿真结果证明了原对偶 PPO 算法及其多线程版本在发射波束成形和相移设计方面的有效性。