Skip to main content
SpringerLink
Log in

A single-player Monte Carlo tree search method combined with node importance for virtual network embedding

  • Published:
Annals of Telecommunications Aims and scope Submit manuscript

Abstract

As a critical technology in network virtualization, virtual network embedding (VNE) focuses on how to allocate physical resources to virtual network requests efficiently. Because the VNE problem is NP-hard, most of the existing approaches are heuristic-based algorithms that tend to converge to a local optimal solution and have a low performance. In this paper, we propose an algorithm that combines the basic Monte Carlo tree search (MCTS) method with node importance to apply domain-specific knowledge. For a virtual network request, we first model the embedding process as a finite Markov decision process (MDP), where each virtual node is embedded in one state in the order of node importance that we design. The shortest-path algorithm is then applied to embed links in the terminal state and return the cost as a part of the reward. Due to the reward delay mechanism of the MDP, the result of link mapping can affect the action selected in the previous node mapping stage, coordinating the two embedding stages. With node importance, domain-specific knowledge can be used in the Expansion and Simulation stages of MCTS to speed up the search and estimate the simulation value more accurately. The experimental results show that, compared with the existing classic algorithms, our proposed algorithm can improve the performance of VNE in terms of the average physical node utilization ratio, acceptance ratio, and long-term revenue to cost ratio.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

References

  1. Bashir AK, Arul R, Basheer S, Raja G, Jayaram R, Qureshi NMF (2019) An optimal multitier resource allocation of cloud RAN in 5G using machine learning. Trans Emerg Telecommun Tech 30(8):e3627

    Google Scholar 

  2. Bashir AK, Ohsita Y, Murata M (2015) Abstraction layer based distributed architecture for virtualized data centers

  3. Chowdhury NMMK, Boutaba R (2009) Network virtualization: state of the art and research challenges. IEEE Commun Mag 47(7):20–26

    Article  Google Scholar 

  4. Mosharaf Kabir Chowdhury NM, Rahman MR, Boutaba R (2009) Virtual network embedding with coordinated node and link mapping. In: IEEE INFOCOM 2009. IEEE, pp 783–791

  5. Wang C, Shanbhag S, Tilman W (2012) Virtual network mapping with traffic matrices. In: IEEE international conference on communications (ICC). IEEE, pp 2717–2722

  6. Cao H, Yang L, Liu Z, Wu M (2016) Exact solutions of vne: a survey. China Commun 13(6):48–62

    Article  Google Scholar 

  7. Cao H, Hu H, Qu Z, Yang L (2018) Heuristic solutions of virtual network embedding: a survey. China Commun 15(3):186–219

    Article  Google Scholar 

  8. Zhang P, Yao H, Liu Y (2016) Virtual network embedding based on the degree and clustering coefficient information. IEEE Access 4:8572–8580

    Article  Google Scholar 

  9. Cheng X, Su S, Zhang Z, Wang H, Yang F, Luo Y, Wang J (2011) Virtual network embedding through topology-aware node ranking. Comput. Commun. Rev 41(2):39–47

    Article  Google Scholar 

  10. Cao H, Zhu Y, Yang L, Zheng G (2017) A efficient mapping algorithm with novel node-ranking approach for embedding virtual networks. IEEE Access 5(1):22054–22066

    Article  Google Scholar 

  11. Zhang P, Yao H, Liu Y (2018) Virtual network embedding based on computing, network, and storage resource constraints. IEEE Internet Things J. 5(5):3298–3304

    Article  Google Scholar 

  12. Zhang P, Yao H, Qiu C, Liu Y (2018) Virtual network embedding using node multiple metrics based on simplified ELECTRE method. IEEE Access 6:37314–37327

    Article  Google Scholar 

  13. Sutton RS, Barto AG (2018) Reinforcement learning: an introduction MIT press

  14. Bashir AK, Ohsita Y, Murata M (2016) Abstraction layer based virtual data center architecture for network function chaining

  15. Korf RE (1985) Depth-first iterative-deepening, vol 27

  16. Swiechowski M, Mandziuk J (2014) Self-adaptation of playing strategies in general game playing, vol 6

  17. Haeri S, Trajković L (2018) Virtual network embedding via Monte Carlo tree search. IEEE Trans Cybern. 48(2):510–521

    Article  Google Scholar 

  18. Soualah O, Fajjari I, Aitsaadi N, Mellouk A (2015) A batch approach for a survivable virtual network embedding based on monte-carlo tree search. In: IFIP/IEEE international symposium on integrated network management (IM). IEEE, pp 36– 43

  19. Yao H, Chen X, Li M, Zhang P, Wang L (2018) A novel reinforcement learning algorithm for virtual network embedding. Neurocomputing 284:1–9

    Article  Google Scholar 

  20. Schadd M.P.D., Winands M.H.M., van den Herik H.J., Chaslot G.M.J.B., Uiterwijk J.W.H.M. (2008) Single-Player Monte-Carlo Tree Search. In: van den Herik H.J., Xu X., Ma Z., Winands M.H.M. (eds), vol 5131. Computers and Games. CG 2008. Lecture Notes in Computer Science Springer, Berlin, Heidelberg, pp 1–12

  21. Fischer A, Botero JF, Beck MT, De Meer H, Hesselbach Xavier (2013) Virtual network embedding: a survey. IEEE Commun Surveys Tuts 15(4):1888–1906

    Article  Google Scholar 

  22. Kennedy J (2010). In: Sammut C., Webb G. I. (eds) Particle Swarm Optimization. Springer US, Boston, MA, pp 760–766. isbn:978-0-387-30164-8

  23. Dorigo M, Birattari M (2010) Ant colony optimization. Springer, Berlin

    Google Scholar 

  24. Goldberg DE (2006) Genetic algorithms. Pearson Education India, Bengaluru

    Google Scholar 

  25. Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science 220 (4598):671–680

    Article  MathSciNet  Google Scholar 

  26. Wang C, Yian S, Zhou L, Peng S, Yuan Y, Huang H (2016) A virtual network embedding algorithm based on hybrid particle swarm optimization. In: Int conf smart comput commun, pp 568–576

  27. Mijumbi R, Gorricho J-L, Serrat J, Claeys M, De Turck F, Latré S (2014) Design and evaluation of learning algorithms for dynamic resource management in virtual networks. In: 2014 IEEE netw oper manag symp, pp 1–9

  28. Yao H, Zhang B, Zhang P, Wu S, Jiang C, Guo S (2018) RDAM: a reinforcement learning based dynamic attribute matrix representation for virtual network embedding. IEEE trans Emerg Top comput

  29. Yu M, Yi Y, Rexford J, Chiang M (2008) Rethinking virtual network embedding: substrate support for path splitting and migration. ACM SIGCOMM Comput Commun Rev 38(2):17–29

    Article  Google Scholar 

  30. Kocsis L, Szepesvári C (2006) Bandit based Monte-Carlo planning

  31. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, et al. (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484

    Article  Google Scholar 

  32. Floyd RW (1962) Algorithm 97: shortest path. Commun. ACM 5(6):345

    Article  Google Scholar 

  33. Zhang Z, Cheng X, Su S, Wang Y, Shuang K, Luo Y (2013) A unified enhanced particle swarm optimization-based virtual network embedding algorithm. Int J Commun Syst 26(8):1054–1073

    Article  Google Scholar 

  34. Calheiros RN, Ranjan R, Beloglazov A, De Rose CAF, Buyya R (2011) CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw Pract Exp 41(1):23–50

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China under Grant Nos. 61702089, the basic scientific research operating fund of central universities under Grant No. N182304021, and the scientific research plan for institutions of higher learning of Hebei province under Grant No. ZD2019306.

Author information

Author notes

    Authors and Affiliations

    1. School of Computer and Communication Engineering, Northeastern University at Qinhuangdao, Qinhuangdao, Hebei, 066004, China

      Guangcong Zheng, Cong Wang, Weijie Shao, Ying Yuan & Zejie Tian

    2. Laboratory of Language Engineering and Computing, Guangdong University of Foreign Studies, Guangzhou, 510006, China

      Sancheng Peng

    3. Department of Computing and Mathematics, Manchester Metropolitan University, Manchester, UK

      Ali Kashif Bashir

    4. School of Electrical Engineering and Computer Science, National University of Science and Technology, Islamabad, (NUST), Pakistan

      Ali Kashif Bashir

    5. Instituto de Telecomunicações, Campus Universitário de Santiago, Aveiro, Portugal

      Shahid Mumtaz

    Authors
    1. Guangcong Zheng
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Cong Wang
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Weijie Shao
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Ying Yuan
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Zejie Tian
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. Sancheng Peng
      View author publications

      You can also search for this author in PubMed Google Scholar

    7. Ali Kashif Bashir
      View author publications

      You can also search for this author in PubMed Google Scholar

    8. Shahid Mumtaz
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Cong Wang.

    Additional information

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Guangcong Zheng and Cong Wang contributed equally to this work and should be considered co-first authors.

    Rights and permissions

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Zheng, G., Wang, C., Shao, W. et al. A single-player Monte Carlo tree search method combined with node importance for virtual network embedding. Ann. Telecommun. 76, 297–312 (2021). https://doi.org/10.1007/s12243-020-00772-5

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s12243-020-00772-5

    Keywords

    Search

    Navigation