Since Network Function Virtualization (NFV) decouples network functions (NFs) from the underlying dedicated hardware and realizes them in the form of software called Virtual Network Functions (VNFs), they are enabled to run in any resource-sufficient virtual machines. A service function chain (SFC) is composed of a sequential set of VNFs. As VNFs are vulnerable to various faults such as software failures, we consider how to deploy both active and standby SFC instances. Given the complexity and unpredictability of the network state, we propose a double deep Q-networks based online SFC placement scheme DDQP. Specifically, DDQP uses deep neural networks to deal with large continuous network state space. In the case of stateful VNFs, we offer constant generated state updates from active instances to standby instances to guarantee seamless redirection after failures. With the goal of balancing the waste of resources and ensuring service reliability, we introduce five progressive schemes of resource reservations to meet different customer needs. Our experimental results demonstrate that DDQP responds rapidly to arriving requests and reaches near-optimal performance. Specifically, DDQP outweighs the state-of-the-art method by 16.30% and 38.51% higher acceptance ratio under different schemes with 82x speedup on average. In order to enhance the integrity of the SFC state transition, we further proposed DDQP+, which extends DDQP by adding the delayed placement mechanism. Compared with DDQP, the design of the DDQP+ algorithm is more reasonable and comprehensive. The experiment results also show that DDQP+ achieved further improvement in multiple performance indicators.

中文翻译：

---

DDQP：在线容错SFC放置的双重深度Q学习方法

由于网络功能虚拟化（NFV）使网络功能（NF）与底层专用硬件分离，并以称为虚拟网络功能（VNF）的软件形式实现它们，因此可以在任何资源充足的虚拟机中运行它们。服务功能链（SFC）由一组顺序的VNF组成。由于VNF容易受到各种故障（例如软件故障）的影响，因此我们考虑如何同时部署活动和备用SFC实例。考虑到网络状态的复杂性和不可预测性，我们提出了一种基于双深度Q网络的在线SFC放置方案DDQP。具体来说，DDQP使用深度神经网络来处理大型连续网络状态空间。对于有状态的VNF，我们提供从活动实例到备用实例的不断生成的状态更新，以确保发生故障后进行无缝重定向。为了平衡资源浪费和确保服务可靠性，我们介绍了五种渐进的资源预留方案，以满足不同的客户需求。我们的实验结果表明，DDQP可以快速响应到达的请求并达到接近最佳的性能。具体而言，在不同方案下，DDQP的接受率要比最新方法高出16.30％和38.51％，平均速度要高出82倍。为了增强SFC状态转换的完整性，我们进一步提出了DDQP +，它通过添加延迟放置机制来扩展DDQP。与DDQP相比，DDQP +算法的设计更加合理，全面。