Network robustness against sequential attacks is significant for complex networks. However, it is generally assumed that complete information of complex networks is obtained and arbitrary nodes can be removed in previous researches. In this paper, a sequential attack in complex networks is modeled as a partial observable Markov decision process (POMDP). Then a reinforcement learning (RL) approach for POMDP is proposed to analyze dynamical robustness of complex networks under sequential attacks, when information of networks is incomplete. According to this approach, an agent can learn to take action by exploiting experiences. To solve the problem of large state space in complex networks, deep $Q$-network algorithm is used to identify most damaging sequential attacks, as deep neural networks can build up progressively abstract representations of state space of complex networks. The performances of proposed approach are analyzed on scale-free networks and small-world networks. According to the numerical simulations, it is found that the RL-based sequential attacks perform better when load distributions are more heterogeneous and local connections are more significant. Furthermore, it is shown that increasing the proportions of observed and attacked nodes improves the performance of RL-based sequential attacks. Finally, the results are verified on the IEEE 300-bus system and the simulation results highlight the damages caused by RL-based sequential attacks.

抵御顺序攻击的网络稳健性对于复杂的网络非常重要。然而，通常假设在先前的研究中可以获得复杂网络的完整信息并且可以删除任意节点。本文将复杂网络中的顺序攻击建模为部分可观察的马尔可夫决策过程（POMDP）。在网络信息不完整的情况下，提出了一种针对POMDP的强化学习方法，以分析复杂网络在有序攻击下的动态鲁棒性。根据这种方法，代理可以通过利用经验来学习采取行动。解决复杂网络中状态空间大的问题$问$网络算法用于识别最具破坏性的顺序攻击，因为深度神经网络可以逐步建立复杂网络状态空间的抽象表示。在无标度网络和小世界网络上分析了该方法的性能。根据数值模拟，发现当负载分布更加异构并且本地连接更加重要时，基于RL的顺序攻击会表现更好。此外，研究表明，增加被观察节点和被攻击节点的比例可以提高基于RL的顺序攻击的性能。最后，在IEEE 300总线系统上对结果进行了验证，仿真结果突出了基于RL的顺序攻击所造成的破坏。