Network Function Virtualization (NFV) introduces a new network architecture framework that evolves network functions, traditionally deployed over dedicated equipment, to software implementations that run on general-purpose hardware. One of the main challenges for deploying NFV is the optimal resource placement of demanded network services in the NFV infrastructure. The virtual network function placement and network embedding can be formulated as a mathematical optimization problem concerned with a set of feasibility constraints that express the restrictions of the network infrastructure and the services contracted. This problem has been reported to be NP-hard, as a result most of the optimization work carried out in the area has focused on designing heuristic and metaheuristic algorithms. Nevertheless, in highly constrained problems, as in this case, inferring a competitive heuristic can be a daunting task that requires expertise. Consequently, an interesting solution is the use of Reinforcement Learning to model an optimization policy. The work presented here extends the Neural Combinatorial Optimization theory by considering constraints in the definition of the problem. The resulting agent is able to learn placement decisions by exploring the NFV infrastructure with the aim of minimizing the overall power consumption. The experiments conducted demonstrate that when the proposed strategy is also combined with heuristics, highly competitive results are achieved using relatively simple algorithms.

中文翻译：

---

使用深度强化学习优化虚拟网络功能放置

网络功能虚拟化 (NFV) 引入了一种新的网络架构框架，该框架将传统上部署在专用设备上的网络功能发展为在通用硬件上运行的软件实现。部署 NFV 的主要挑战之一是在 NFV 基础设施中所需网络服务的最佳资源放置。虚拟网络功能放置和网络嵌入可以被表述为一个数学优化问题，该问题涉及一组表达网络基础设施和签约服务限制的可行性约束。据报道，这个问题是 NP-hard 问题，因此该领域进行的大部分优化工作都集中在设计启发式和元启发式算法上。然而，在高度受限的问题中，在这种情况下，推断竞争性启发式可能是一项艰巨的任务，需要专业知识。因此，一个有趣的解决方案是使用强化学习来建模优化策略。此处介绍的工作通过考虑问题定义中的约束扩展了神经组合优化理论。由此产生的代理能够通过探索 NFV 基础设施来学习布局决策，目的是最大限度地降低整体功耗。进行的实验表明，当所提出的策略也与启发式方法相结合时，使用相对简单的算法可以获得极具竞争力的结果。一个有趣的解决方案是使用强化学习来建模优化策略。此处介绍的工作通过考虑问题定义中的约束扩展了神经组合优化理论。由此产生的代理能够通过探索 NFV 基础设施来学习布局决策，目的是最大限度地降低整体功耗。进行的实验表明，当所提出的策略也与启发式方法相结合时，使用相对简单的算法可以获得极具竞争力的结果。一个有趣的解决方案是使用强化学习来建模优化策略。此处介绍的工作通过考虑问题定义中的约束扩展了神经组合优化理论。由此产生的代理能够通过探索 NFV 基础设施来学习布局决策，目的是最大限度地降低整体功耗。进行的实验表明，当所提出的策略也与启发式方法相结合时，使用相对简单的算法可以获得极具竞争力的结果。