Skip to main content
SpringerLink
Log in

Adaptive Workload Forecasting in Cloud Data Centers

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Forecasting on different levels of the management system of a cloud data center has received increased attention due to its significant impact on the cloud services quality. Making accurate forecasts, however, is challenging due to the non-stationary workload and intrinsic complexity of the management system of a cloud data center. It is possible to prevent excessive resource allocation and service level agreement violations through workload forecasting for virtual machines and containers. In this paper, the authors propose the adaptive forecasting model and corresponding adaptive forecasting methods to apply in the management system of a cloud data center for workload forecasting, ensuring compliance with the service level agreement and power consumption decrease. The authors consider six alternative forecasting methods and 77 training data windows on each management step to determine the best combination of methods and the training set size that generates a most accurate forecast, thereby adapting to the current state of the physical or virtual server in a cloud data center. Through the comprehensive analysis, the authors also evaluate the proposed adaptive forecasting methods using real-world workload traces Bitbrains and demonstrate that combined forecasting methods outperform the individual forecasting methods significantly in terms of forecasting accuracy measured by Mean Absolute Percentage Error.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Armstrong, J.S. (ed.): Principles of Forecasting: A Handbook for Researchers and Practitioners, vol. 30. Springer Science & Business Media, New York (2001)

  2. Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., et al.: Xen and the art of virtualization. In: ACM SIGOPS Operating Systems Review, vol. 37, No. 5, pp. 164–177. ACM (2003)

  3. Bates, J.M., Granger, C.W.J.: The combination of forecasts. Oper. Res. Q. 20(4), 451–468 (1969). https://doi.org/10.1057/jors.1969.103

    Article  Google Scholar 

  4. Bidyuk, P.I., Meniailenko, O.S., Polovtsev, O.V.: Methods of Forecasting, vol. 1, p. 308. Alma-mater, Luhansk (2008)

    Google Scholar 

  5. Box, G.E., Jenkins, G.M., Reinsel, G.C., Ljung, G.M.: Time Series Analysis: Forecasting and Control, 5th edn. Wiley, Hoboken (2015)

    MATH  Google Scholar 

  6. Che, J.: Optimal sub-models selection algorithm for combination forecasting model. Neurocomputing. 151, 364–375 (2015)

    Article  Google Scholar 

  7. Chen, G., He, W., Liu, J., Nath, S., Rigas, L., Xiao, L., Zhao, F.: Energy-aware server provisioning and load dispatching for connection-intensive internet services. NSDI. 8, 337–350 (2008)

    Google Scholar 

  8. Clemen, R.: Combining forecasts: a review and annotated bibliography with discussion. Int. J. Forecast. 5, 559–608 (1989). https://doi.org/10.1016/0169-2070(89)90012-5

    Article  Google Scholar 

  9. da Rosa Righi, R., Lehmann, M., Gomes, M.M., Nobre, J.C., da Costa, C.A., Rigo, S.J., et al.: A survey on global management view: toward combining system monitoring, resource management, and load prediction. J. Grid. Computing. 1–30 (2019)

  10. Dabbagh, M., Hamdaoui, B., Guizani, M., Rayes, A.: Energy-efficient resource allocation and provisioning framework for cloud data centers. IEEE Trans. Netw. Serv. Manag. 12(3), 377–391 (2015)

    Article  Google Scholar 

  11. De Livera, A.M., Hyndman, R.J., Snyder, R.D.: Forecasting time series with complex seasonal patterns using exponential smoothing. J. Am. Stat. Assoc. 106(496), 1513–1527 (2011)

    Article  MathSciNet  Google Scholar 

  12. Dinda, P.A.: Design, implementation, and performance of an extensible toolkit for resource prediction in distributed systems. IEEE Trans. Parallel Distrib. Syst. 17(2), 160–173 (2006)

    Article  Google Scholar 

  13. Farahnakian, F., Liljeberg, P., Plosila, J.: LiRCUP: linear regression based CPU usage prediction algorithm for live migration of virtual machines in data centers. In: Software Engineering and Advanced Applications (SEAA), 2013 39th EUROMICRO Conference on, pp. 357–364. IEEE (2013)

  14. Feitelson, D.G.: Workload Modeling for Computer Systems Performance Evaluation. Cambridge University Press, Cambridge (2015)

    Book  Google Scholar 

  15. Gardner Jr., E.S., McKenzie, E.D.: Forecasting trends in time series. Manag. Sci. 31(10), 1237–1246 (1985)

    Article  Google Scholar 

  16. Gross, G., Galiana, F.D.: Short-term load forecasting. Proc. IEEE. 75(12), 1558–1573 (1987)

    Article  Google Scholar 

  17. GWA-T-12 Bitbrains [Online] Available from: http://gwa.ewi.tudelft.nl/datasets/gwa-t-12-bitbrains. Accessed 27 Dec 2018

  18. Holt, C.C.: Forecasting seasonals and trends by exponentially weighted moving averages. Int. J. Forecast. 20(1), 5–10 (2004)

    Article  Google Scholar 

  19. Hyndman, R.J., Khandakar, Y.: Automatic time series forecasting: the forecast package for R. J. Stat. Softw. 27(1), 1–22 (2008). Retrieved from https://www.jstatsoft.org/article/view/v027i03

    Google Scholar 

  20. Hyndman, R.J., Athanasopoulos, G.: Forecasting: Principles and Practice. OTexts, Heathmont (2018)

    Google Scholar 

  21. Hyndman, R., Bergmeir, C., Caceres, G., Chhay, L., O’Hara-Wild, M., Petropoulos, F., Razbash, S., Wang, E., Yasmeen, F.: “_forecast: forecasting functions for time series and linear models_”. R package version 8.3. (2018). http://pkg.robjhyndman.com/forecast

  22. Hyndman, R., Koehler, A.B., Ord, J.K., Snyder, R.D.: Forecasting with Exponential Smoothing: the State Space Approach. Springer Science & Business Media, Berlin (2008)

    Book  Google Scholar 

  23. Islam, S., Keung, J., Lee, K., Liu, A.: Empirical prediction models for adaptive resource provisioning in the cloud. Futur. Gener. Comput. Syst. 28(1), 155–162 (2012)

    Article  Google Scholar 

  24. Ivakhnenko, A.G.: The group method of data handling (GMDH). Automation. 3, 57–83 (1968)

    Google Scholar 

  25. Jorgensen, M.: Experience with the accuracy of software maintenance task effort prediction models. IEEE Trans. Softw. Eng. 21(8), 674–681 (1995)

    Article  Google Scholar 

  26. Montgomery, D.C., Peck, E.A., Geoffrey Vining, G.: Introduction to Linear Regression Analysis. Wiley, New York (2015)

    MATH  Google Scholar 

  27. Naseera, S., Rajini, G.K., Reddy, P.S.K.: Host CPU load prediction using statistical algorithms a comparative study. Int. J. Comput. Technol. Appl. 9(12), 5577–5582 (2016)

    Google Scholar 

  28. Naseera, S., Rajini, G.K., Prabha, N.A., Abhishek, G.: A comparative study on CPU load predictions in a computational grid using artificial neural network algorithms. Indian J. Sci. Technol. 8(35), (2015)

  29. Nowotarski, J., Liu, B., Weron, R., Hong, T.: Improving short term load forecast accuracy via combining sister forecasts. Energy. 98, 40–49 (2016)

    Article  Google Scholar 

  30. Padala, P., Hou, K.Y., Shin, K.G., Zhu, X., Uysal, M., Wang, Z., et al.: Automated control of multiple virtualized resources. In: Proceedings of the 4th ACM European Conference on Computer systems, pp. 13–26. ACM (2009)

  31. Park, K., Pai, V.S.: CoMon: a mostly-scalable monitoring system for PlanetLab. ACM SIGOPS Oper. Syst. Rev. 65–47 (2006)

  32. Core Team, R.: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2018). https://www.R-project.org/

    Google Scholar 

  33. R-3.4.4 for Windows (32/64 bit). https://cran.r-project.org/bin/windows/base/old/3.4.4/

  34. Shen, S., van Beek, V., Iosup, A.: Statistical characterization of business-critical workloads hosted in cloud datacenters. In: Cluster, Cloud and Grid Computing (CCGrid), 2015 15th IEEE/ACM International Symposium on, pp. 465–474. IEEE (2015)

  35. Singh, S., Chana, I.: A survey on resource scheduling in cloud computing: issues and challenges. J Grid Comput. 14(2), 217–264 (2016)

    Article  Google Scholar 

  36. Tang, X.: Large-scale computing systems workload prediction using parallel improved LSTM neural network. IEEE Access. 7, 40525–40533 (2019)

    Article  Google Scholar 

  37. Tang, X., Liao, X., Zheng, J., Yang, X.: Energy efficient job scheduling with workload prediction on cloud data center. Clust. Comput. 21(3), 1581–1593 (2018)

    Article  Google Scholar 

  38. Tang, Z., Mo, Y., Li, K., Li, K.: Dynamic forecast scheduling algorithm for virtual machine placement in cloud computing environment. J. Supercomput. 70(3), 1279–1296 (2014)

    Article  Google Scholar 

  39. Telenyk, S., Zharikov, E., Rolik, O.: Architecture and conceptual bases of cloud IT infrastructure management. In: Advances in Intelligent Systems and Computing, pp. 41–62. Springer, Cham (2017)

    Chapter  Google Scholar 

  40. Wallis, K.F.: Combining forecasts–forty years later. Appl. Financ. Econ. 21(1–2), 33–41 (2011)

    Article  Google Scholar 

  41. Xiao, Z., Song, W., Chen, Q.: Dynamic resource allocation using virtual machines for cloud computing environment. IEEE Trans. Parallel Distrib. Syst. 24(6), 1107–1117 (2013)

    Article  Google Scholar 

  42. Xue, J., Yan, F., Birke, R., Chen, L.Y., Scherer, T., Smirni, E.: Practise: robust prediction of data center time series. In: Network and Service Management (CNSM), 2015 11th International Conference on, pp. 126–134. IEEE (2015)

  43. Yoo, W., Sim, A.: Time-series forecast modeling on high-bandwidth network measurements. J Grid Comput. 14(3), 463–476 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Politekhnichna St, 41, Academic building № 18, Kyiv, 03056, Ukraine

    Eduard Zharikov, Sergii Telenyk & Petro Bidyuk

  2. Cracow University of Technology, Cracow, Poland

    Sergii Telenyk

Authors
  1. Eduard Zharikov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sergii Telenyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Petro Bidyuk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eduard Zharikov.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zharikov, E., Telenyk, S. & Bidyuk, P. Adaptive Workload Forecasting in Cloud Data Centers. J Grid Computing 18, 149–168 (2020). https://doi.org/10.1007/s10723-019-09501-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-019-09501-2

Keywords

Search

Navigation