The smart ammunition formation (SAF) system model usually has the characteristics of complexity, time variation, and nonlinearity. With the consideration of random factors, such as sensor error and environmental disturbance, the system model cannot be modeled accurately. To deal with this problem, this paper investigated an intelligent deep -network- (DQN-) based control algorithm for the SAF collaborative control, which deals with the high dynamics and uncertainty in the SAF flight environment. In the environment description of the SAF, we built a dynamic model to represent the system joint states, which referred to the smart ammunition’s velocity, the trajectory inclination angle, the ballistic deflection angle, and the relative position between different formation nodes. Next, we describe the SAF collaborative control process as a Markov decision process (MDP) with the application of the reinforcement learning (RL) technique. Then, the basic framework -imitation action-selecting strategy and the algorithm details were developed to address the SAF control problem based on the DQN scheme. Finally, the numerical simulation was carried out to verify the effectiveness and portability of the DQN-based algorithm. The average total reward curve showed a reasonable convergence, and the relative kinematic relationship among the formation nodes met the requirements of the controller design. It illustrated that the DQN-based algorithm obtained a novel performance in the SAF collaborative control.

中文翻译：

---

基于深度网络的智能弹药编队协同控制研究

智能弹药编队（SAF）系统模型通常具有复杂性、时变性和非线性等特点。考虑到传感器误差和环境干扰等随机因素，系统模型无法准确建模。针对这一问题，本文研究了一种智能深度——基于网络（DQN）的 SAF 协同控制控制算法，处理 SAF 飞行环境中的高动态性和不确定性。在SAF的环境描述中，我们建立了一个动态模型来表示系统联合状态，该模型涉及智能弹药的速度、弹道倾角、弹道偏转角以及不同编队节点之间的相对位置。接下来，我们将 SAF 协同控制过程描述为应用强化学习 (RL) 技术的马尔可夫决策过程 (MDP)。然后，基本框架——开发了模仿动作选择策略和算法细节，以解决基于 DQN 方案的 SAF 控制问题。最后通过数值仿真验证了基于DQN算法的有效性和可移植性。平均总奖励曲线呈现出合理的收敛性，编队节点间的相对运动学关系满足控制器设计要求。说明基于DQN的算法在SAF协同控制中获得了新颖的性能。