Secure transmission is essential for future non-orthogonal multiple access (NOMA) system. This paper investigates relay–antenna selection to enhance physical-layer security of cooperative NOMA system in the presence of an eavesdropper, where multiple antennas are deployed at the relays, the users, and the eavesdropper. In order to reduce expense on radio frequency chains, selection combining (SC) is employed at both the relays and the users, whilst the eavesdropper employs either maximal-ratio combining or SC to process the received signals. Under the condition that the channel state information of the eavesdropping channel is available or unavailable, two effective relay–antenna selection schemes are proposed. Additionally, the closed-form expressions of secrecy outage probability (SOP) are derived for the proposed relay–antenna selection schemes. In order to gain more deep insights on the derived results, the asymptotic performance of the derived SOP is analyzed. In simulations, it is demonstrated that the theoretical results match well with the simulation results and the SOP of the proposed schemes is less than that of the conventional orthogonal multiple access scheme obviously.

安全传输对于未来的非正交多址（NOMA）系统至关重要。本文研究了中继天线选择，以在存在窃听者的情况下增强协作 NOMA 系统的物理层安全性，其中在中继、用户和窃听者处部署了多个天线。为了减少射频链的开销，中继和用户都采用选择合并（SC），而窃听者采用最大比合并或SC来处理接收到的信号。在窃听信道的信道状态信息可用或不可用的情况下，提出了两种有效的中继天线选择方案。此外，针对所提出的中继天线选择方案，导出了保密中断概率 (SOP) 的封闭式表达式。为了更深入地了解衍生结果，分析了衍生 SOP 的渐近性能。仿真结果表明，理论结果与仿真结果吻合较好，所提方案的SOP明显低于传统正交多址接入方案。