Skip to main content
SpringerLink
Log in

HFOS\(_L\): hyper scale fast optical switch-based data center network with L-level sub-network

  • Published:
Telecommunication Systems Aims and scope Submit manuscript

Abstract

The ever-expanding growth of internet traffic enforces deployment of massive Data Center Networks (DCNs) supporting high performance communications. Optical switching is being studied as a promising approach to fulfill the surging requirements of large scale data centers. The tree-based optical topology limits the scalability of the interconnected network due to the limitations in the port count of optical switches and the lack of optical buffers. Alternatively, buffer-less Fast Optical Switch (FOS) was proposed to realize the nanosecond switching of optical DCNs. Although FOSs provide nanosecond optical switching, they still suffer from port count limitations to scale the DCN. To address the issue of scaling DCNs to more than two million servers, we propose the hyper scale FOS-based L-level DCNs (HFOS\(_L\)) which is capable of building large networks with small radix switches. The numerical analysis shows L of 4 is the optimal level for HFOS\(_L\) to obtain the lowest cost and power consumption. Specifically, under a network size of 160,000 servers, HFOS\(_4\) saves 36.2% in cost compared with the 2-level FOS-based DCN, while achieves 60% improvement for cost and 26.7% improvement for power consumption compared with Fat tree. Moreover, a wide range of simulations and analyses demonstrate that HFOS\(_4\) outperforms state-of-art FOS-based DCNs by up to 40% end-to-end latency under DCN size of 81920 servers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Availability of data and material

Not applicable

References

  1. Global data center IP traffic 2013-2021 Statista. [Online] Available: https://www.statista.com/statistics/227246/global-datacenter-ip-traffic-development-forecast.

  2. Cheng, Q., Glick, M., & Bergman, K. (2020). Optical interconnection networks for high-performance systems. In Optical Fiber Telecommunications VII (pp. 785–825). Academic Press.

  3. Ghiasi, A. (2015). Large data centers interconnect bottlenecks. Optics Express, 23(3), 2085–2090. https://doi.org/10.1364/OE.23.002085

    Article  Google Scholar 

  4. Ballani, H., Costa, P., Behrendt, R., Cletheroe, D., Haller, I., Jozwik, K., ... & Williams, H. (2020). Sirius: A flat datacenter network with nanosecond optical switching. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication (pp. 782–797).

  5. Minkenberg, C., Farrington, N., Zilkie, A., Nelson, D., Lai, C. P., Brunina, D., Byrd, J., Chowdhuri, B., Kucharewski, N., Muth, K., Nagra, A., Rodriguez, G., Rubi, D., Schrans, T., Srinivasan, P., Wang, Y., Yeh, C., & Rickman, A. (2018). Reimagining datacenter topologies with integrated silicon photonics. Journal of Optical Communications and Networking, 10(7), B126–B139. https://doi.org/10.1364/JOCN.10.00B126

    Article  Google Scholar 

  6. Testa, F., & Pavesi, L. (Eds.). (2017). Optical switching in next generation data centers. Berlin: Springer.

    Google Scholar 

  7. Xu, M., Diakonikolas, J., Modiano, E., & Subramaniam, S. (2019). A hierarchical WDM-based scalable data center network architecture. In ICC 2019-2019 IEEE International Conference on Communications (ICC) (pp. 1–7).

  8. Prifti, K., Santos, R., Shin, J., Kim, H., Tessema, N., Stabile, P., Kleijn, S., Augustin, L., Jung, H., Park, S., Baek, Y., Hyun, S., & Calabretta, N. (2020). All-optical cross-connect switch for data center network application. In 2020 Optical Fiber Communications Conference and Exhibition (OFC) (pp. 1–3).

  9. Foerster, K., & Schmid, S. (2019). Survey of reconfigurable data center networks: Enablers, algorithms, complexity. SIGACT News, 50(2), 62–79. https://doi.org/10.1145/3351452.3351464

    Article  Google Scholar 

  10. Chen, K., Singla, A., Singh, A., Ramachandran, K., Xu, L., Zhang, Y., Wen, X., & Chen, Y. (2014). OSA: An optical switching architecture for data center networks with unprecedented flexibility. IEEE/ACM Transaction On Networking, 22(2), 498–511. https://doi.org/10.1109/TNET.2013.2253120

    Article  Google Scholar 

  11. Nance Hall, M., Foerster, K. T., Schmid, S., & Durairajan, R. (2021). A survey of reconfigurable optical networks. Optical Switching and Networking. https://doi.org/10.1016/j.osn.2021.100621

    Article  Google Scholar 

  12. Yin, Y., Proietti, R., Ye, X., Nitta, C., Akella, V., & Yoo, S. (2013). LIONS: An AWGR-based low-latency optical switch for high-performance computing and data centers. IEEE Journal of Selected Topic Quantum Electron., 19(2), 3600409–3600409. https://doi.org/10.1109/JSTQE.2012.2209174

    Article  Google Scholar 

  13. Kachris, C., & Tomkos, I. (2013). Power consumption evaluation of all-optical data center networks. Cluster Computing, 16, 611–623. https://doi.org/10.1007/s10586-012-0227-6

    Article  Google Scholar 

  14. Chen, K., Wen, X., Ma, X., Chen, Y., Xia, Y., Hu, C., Dong, Q., & Liu, Y. (2017). Toward a scalable, fault-tolerant, high-performance optical data center architecture. IEEE/ACM Transaction On Networking, 25(4), 2281–2294. https://doi.org/10.1109/TNET.2017.2688376

    Article  Google Scholar 

  15. Keykhosravi, K., Rastegarfar, H., & Agrell, E. (2018). Multicast scheduling of wavelength-tunable, multiqueue optical data center switches. Journal of Optical Communications and Networking, 10(4), 353–364. https://doi.org/10.1364/JOCN.10.000353

    Article  Google Scholar 

  16. Mellette, W. M., Das, R., Guo Y., McGuinness, R., Snoeren, A. C., & Porter, G. (2020). Expanding across time to deliver bandwidth efficiency and low latency. 17th USENIX Symposium on Networked Systems Design and Implementation \((\)NSDI 20\()\), (pp. 1–18).

  17. Johannes, Z., Wolfgang, K., & Andreas, B. (2021). What you need to know about optical circuit reconfigurations in datacenter networks. 2021 33th International Teletraffic Congress (ITC-33). 1–9.

  18. Peng, L., Xiaoshan, Y., Huaxi, G., & Yunfeng, L. (2021). FlexNet: A optical switching architecture for optical data center networks. In 2021 19th International Conference on Optical Communications and Networks (ICOCN). 1-3. https://doi.org/10.1109/ICOCN53177.2021.9563652

  19. Yan, F., Xue, X., & Calabretta, N. (2018). HiFOST: A scalable and low-latency hybrid data center network architecture based on flow-controlled fast optical switches. IEEE/OSA Journal of Optical Communications and Networking, 10(7), 1–14. https://doi.org/10.1364/JOCN.10.0000B1

    Article  Google Scholar 

  20. Xi, K., Kao, Y. H., & Chao, H. J. (2013). A petabit bufferless optical switch for data center networks. Optical interconnects for future data center networks (pp. 135–154). New York, NY: Springer.

    Chapter  Google Scholar 

  21. Bao, J., Dong, D., Zhao, B., & Huang, S. (2019). HyFabric : Minimizing FCT in optical and electrical hybrid data center networks. In Proceedings of the ACM SIGCOMM 2019 Conference Posters and Demos (SIGCOMM Posters and Demos ’19). 57–59. https://doi.org/10.1145/3342280.3342306

  22. Bakopoulos, P., Christodoulopoulos, K., Landi, G., Aziz, M., Zahavi, E., Gallico, D., & Avramopoulos, H. (2018). NEPHELE: An end-to-end scalable and dynamically reconfigurable optical architecture for application-aware SDN cloud data centers. IEEE Communications Magazine, 56(2), 178–188. https://doi.org/10.1109/MCOM.2018.1600804

    Article  Google Scholar 

  23. Sato, K. I., Hasegawa, H., Niwa, T., & Watanabe, T. (2013). A large-scale wavelength routing optical switch for data center networks. IEEE Communications Magazine, 51(9), 46–52. https://doi.org/10.1109/MCOM.2013.6588649

    Article  Google Scholar 

  24. Wang, K., Zhao, L., Gu, H., Yu, X., Wu, G., & Cai, J. (2015). ADON: A scalable AWG-based topology for datacenter optical network. Optical and Quantum Electronics, 47(8), 2541–2554. https://doi.org/10.1007/s11082-015-0136-z

    Article  Google Scholar 

  25. Chen, K., Wen, X., Ma, X., Chen, Y., Xia, Y., Hu, C., Dong, Q., & Liu, Y. (2017). Toward a scalable, fault-tolerant, high-performance optical data center architecture. IEEE/ACM Transactions on Networking, 25(4), 2281–2294. https://doi.org/10.1109/TNET.2017.2688376

    Article  Google Scholar 

  26. Yan, F., Miao, W., Raz, O., & Calabretta, N. (2017). Opsquare: A flat DCN architecture based on flow-controlled optical packet switches. IEEE/OSA Journal of Optical Communications and Networking, 9(4), 291–303. https://doi.org/10.1364/JOCN.9.000291

    Article  Google Scholar 

  27. Khani, E., Yan, F., Guo, X., & Calabretta, N. (2019). Theoretical analysis on multiple layer fast optical switch based data center network architecture. In 2019 24th OptoElectronics and Communications Conference (OECC) and 2019 International Conference on Photonics in Switching and Computing (PSC) (pp. 1–3).

  28. Yan, F., Xue, X., Pan, B., Guo, X., & Calabretta, N. (2018). FOScube: a scalable data center network architecture based on multiple parallel networks and fast optical switches. In 2018 European Conference on Optical Communication (ECOC) (pp. 1–3).

  29. Levi, C., & Segal, M. (2021). Avoiding bottlenecks in networks by short paths. Telecommunication Systems: Modelling, Analysis, Design and Management, 76(4), 491–503. https://doi.org/10.1007/s11235-020-00720-7

    Article  Google Scholar 

  30. Miao, W., Luo, J., Di Lucente, S., Dorren, H., & Calabretta, N. (2014). Novel flat datacenter network architecture based on scalable and flow-controlled optical switch system. Optics Express, 22(3), 2465–2472. https://doi.org/10.1364/OE.22.002465

    Article  Google Scholar 

  31. Benson, T., Akella, A., & Maltz, D. A. (2010, November). Network traffic characteristics of data centers in the wild. In Proceedings of the 10th ACM SIGCOMM conference on Internet measurement (pp. 267–280).

  32. Benson, T., Anand, A., Akella, A., & Zhang, M. (2010). Understanding data center traffic characteristics. ACM SIGCOMM Computer Communication Review, 40(1), 92–99.

    Article  Google Scholar 

  33. Noormohammadpour, M., & Raghavendra, C. S. (2017). Datacenter traffic control: Understanding techniques and tradeoffs. IEEE Communications Surveys & Tutorials, 20(2), 1492–1525. https://doi.org/10.1109/COMST.2017.2782753

    Article  Google Scholar 

  34. Kandula, S., Sengupta, S., Greenberg, A., Patel, P., & Chaiken, R. (2009). The nature of data center traffic: measurements & analysis. IMC ’09: Proceedings of the 9th ACM SIGCOMM conference on Internet measurement, 9, (pp. 202–208).

  35. Tutorial on Internet Monitoring and PingER at SLAC. [Online]. Available:www.slac.stanford.edu/comp/net/wanmon/tutorial.html (accessed Aug. 09, 2020).

  36. Categories-Elpeus Technology. [Online]. Available: http://www.elpeus.com/categories.

Download references

Funding

Not applicable

Author information

Authors and Affiliations

  1. Sharif University of Technology, Tehran, Iran

    Elham Khani, Shaahin Hessabi & Somayyeh Koohi

  2. Eindhoven University of Technology, Eindhoven, The Netherlands

    Fulong Yan & Nicola Calabretta

Authors
  1. Elham Khani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shaahin Hessabi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Somayyeh Koohi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fulong Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nicola Calabretta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaahin Hessabi.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

Code availability

Not applicable

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khani, E., Hessabi, S., Koohi, S. et al. HFOS\(_L\): hyper scale fast optical switch-based data center network with L-level sub-network. Telecommun Syst 80, 397–411 (2022). https://doi.org/10.1007/s11235-022-00905-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11235-022-00905-2

Keywords

Search

Navigation