Engineering advance
A survey on hyperparameters optimization algorithms of forecasting models in smart grid

https://doi.org/10.1016/j.scs.2020.102275Get rights and content

Highlights

  • Comparison of hyperparameters optimization, error, forecasting and preprocessing methods.

  • We critically analyzed data preprocessing models and highlighted important findings.

  • A survey of existing survey papers (ESPs) is conducted·Recency score of ESPs is computed based on number of recent papers reviewed in them.

  • Future research directions are discussed in detail.

Abstract

Forecasting in the smart grid (SG) plays a vital role in maintaining the balance between demand and supply of electricity, efficient energy management, better planning of energy generation units and renewable energy sources and their dispatching and scheduling. Existing forecasting models are being used and new models are developed for a wide range of SG applications. These algorithms have hyperparameters which need to be optimized carefully before forecasting. The optimized values of these algorithms increase the forecasting accuracy up to a significant level. In this paper, we present a brief literature review of forecasting models and the optimization methods used to tune their hyperparameters. In addition, we have also discussed the data preprocessing methods. A comparative analysis of these forecasting models, according to their hyperparameter optimization, error methods and preprocessing methods, is also presented. Besides, we have critically analyzed the existing optimization and data preprocessing models and highlighted the important findings. A survey of existing survey papers is also presented and their recency score is computed based on the number of recent papers reviewed in them. By recent, we mean that the year in which a survey paper is published and its previous three years. Finally, future research directions are discussed in detail.

Introduction

The advancement in technology and increased usage of smart devices bring about the concept of big data. The production of data is growing rapidly and according to Munshi and Mohamed (2017), the volume of big data would increase by the factor of 300 in the upcoming years. The cost of data storage has also been reduced which is paving the way of storing more data and use it in the future. It has become a new focus for the researchers while playing a very important role in engineering, science and technology domains to acquire the efficient solution of the problems. In SG, a huge volume of data is being gathered and stored from sensors, smart meters and other smart devices which can be used for efficient planning and forecasting. So, forecasting using big data has become the new hot topic in this domain. The rapid increment in electricity consumption, intermittent nature of renewable energy sources (RESs) and fluctuations in demand are serious issues of a power system. The core aims of SG are to achieve the balance between demand and supply of electricity, increase the grid's reliability and efficiency and make grid environment friendly. Forecasting, in this regard, plays a very important role. It enables the utility to plan and organize the future decisions related to power generation, electricity price and coordination of electricity generating units and get maximum benefits out of them. Electricity load and price forecasting have gained great attention from researchers in this area as these two factors have a great influence on maintaining the stability of the grid. Additionally, forecasting of power failure, stability of transformers and power network, anomalies, blackouts and energy generation forecasting of RES are also studied and their solutions are provided in the literature.

The prediction of future values of SG components has a great importance as they play an important role as an input to the current decisions (Yusof & Mustaffa, 2016). For example, the knowledge of load demand in the future can play a very important role to set the electricity price values. It is also important for the utility to make different policies related to energy. Several decisions related to power, based on the information of future load. Similarly, a consumer can use the forecasted values of electricity prices and change its load consumption pattern accordingly. In recent years, researchers have proposed a large number of forecasting models for accurate load and price prediction. They can be used in the maintenance of power networks, better scheduling of energy generators, continuous energy provision to the consumers, achieve stability in demand and supply of electricity and maintain the grid reliability. Moreover, effective planning and decision making can save millions of dollars which are very important for the economical growth of a company as well as country.

In literature, researchers have proposed several forecasting methods to predict the load and price of electricity. Fig. 1 shows the classification of forecasting algorithms. Support vector machine (SVM) and neural networks (NNs) are commonly used forecasting algorithms by researchers to make predictions in the SG area. Different variants of these algorithms are also available and used. Bayesian networks (BN) are also one of the frequently used forecasting methods. In addition to these three types, several other forecasting techniques are also implemented for the prediction in SG. In forecasting algorithms, the accuracy is greatly affected by their hyperparameters. So, the values of these parameters should be chosen carefully.

In this paper, we have presented a survey of algorithms used for the optimization of hyperparameters in SG. Their values vary from problem to problem and need accurate optimized values for correct prediction. Inefficient optimization of these parameters results in poor accuracy results and a model that could be the best choice for a forecasting model, does not perform well. To overcome this issue, different optimization methods are applied to optimize these parameters. Fig. 1 shows the optimization methods commonly used for parameters optimization of forecasting models. Here, grid search, gradient descent and cross validation are the frequently used methods. Nature-inspired methods are also proposed by researchers to efficiently optimize these parameters. In the existing literature (Afshin, Sadeghian, & Raahemifar, 2007; Han, Jiang, Ling, & Su, 2019; Jiang et al., 2013; Yusof & Mustaffa, 2016; Zhang & Suganthan, 2016), similar surveys are presented. However, Yusof and Mustaffa (2016) and Afshin et al. (2007) surveyed the hyperparameter tuning methods of the least square support vector machine (LSSVM) only. Similarly, in Zhang and Suganthan (2016), randomized algorithms are surveyed for tuning NNs, Han et al. (2019) presents the survey of metaheuristic algorithms to train random single-hidden layer feedforward neural network (RSLFN), and Jiang et al. (2013) contains the survey of heuristic algorithms for tuning SVM. In our survey, we have reviewed both nature-inspired and statistical methods to tune the hyperparameters of SVM, NNS, BNs and their variants. The contributions of our survey are as follows:

  • A detailed review of forecasting models (from 2014 and onwards) and optimization methods used to tune hyperparameters of these models is presented.

  • Data preprocessing methods used in these studies are also discussed.

  • All the forecasting models are critically analyzed and future research directions are also presented.

  • In related work section, a survey of similar survey papers is presented and their recency score is also computed.

The rest of the paper is organized as follows: Section 2 contains similar work and in Section 3, we have discussed the proposed framework of a forecasting model that contains all the necessary steps for data forecasting, from data gathering to the final output phase. Section 4 contains a detailed discussion on forecasting techniques and optimizations methods used to optimize the values of their hyperparameters. In Section 5, common data preprocessing methods are discussed. Section 6 contains the critical analysis and findings of this survey. In Section 7, future directions related to hyperparameter optimizers are discussed. Finally, paper is concluded in Section 8.

Section snippets

Related work

Hyperparameter optimization is considered very important for the forecasting accuracy of algorithms. Researchers are using the already existing optimizers and also proposing new optimization algorithms for their tuning. Improvement of forecasting accuracy is an ongoing research area. So, to summarize this research and provide compact information to the readers, survey papers are published. The overview of some of the most related literature to our work is given below.

Forecasting model

Forecasting can be defined as a process in which current and past data values are analyzed to predict future values. In SG, electricity price and demand forecasting are of great importance. With the advancement in technology and increased Internet of things applications, a huge amount of data is gathered. This data is used to predict future energy demand, price, fault detection, electricity theft, etc. Researchers are actively working in this area and proposing efficient forecasting models with

Classification of tuning methods

In SG, the hyperparameters of forecasting algorithms are tuned using different optimizations techniques. The commonly used methods are grid search, cross validation, gradient descent and naive Bayes (NB). Nature-inspired heuristic algorithms are also getting popular in this field. Fig. 6 contains detailed classification of these algorithms. They are classified into two major categories: nature-inspired algorithms and other statistical methods.

Data preprocessing

In the previous section, we have discussed optimization methods for the hyperparameter tuning of forecasting algorithms in detail. Another important factor which affects the forecasting accuracy of these methods is the quality of data. A data set mostly includes noise in it, and it needs to be cleaned before using it for forecasting. In this section, we are going to discuss some data preprocessing methods, used in literature for data cleaning.

Wang, Xu, et al. (2017) have proposed a novel

Critical analysis

In this section, the analysis of the frequently used hyperparameter optimization methods for forecasting algorithms used in the SG domain is presented. Hyperparameter training is very important for the efficient forecasting. It trains the model according to the dataset and tuning these parameters improves the forecasting accuracy significantly. So, we have discussed the tuning methods used by researchers in recent years. These methods are compared in terms of their performance in optimization.

Future directions

The study of existing literature depicts the improvement in hyperparameter tuning is a continuous research area. Some future directions and challenges are identified from the available literature. In this section, we discuss some important future directions for hyperparameter optimizers for their improvement.

  • 1.

    The SG is evolving with each passing day and new actors are being integrated into it. So, it will require new models and applications based on forecasting algorithms e.g. ANN. These newly

Conclusion

This paper presents a brief and comprehensive survey of optimization techniques used for the optimization of hyperparameters of the forecasting model in SG. From literature, it is observed that the grid search and cross validation techniques are commonly used methods but as the size of the dataset increases, they require more computational time. On the other hand, researchers have applied nature-inspired heuristic optimization techniques to optimize these parameters. These techniques work

Conflict of interest

The authors declare that there is no conflict of interest.

Rabiya Khalid received the MCS degree from Mirpur University of Science and Technology, Mirpur (Azad Kashmir), Pakistan, in 2014, and the M.S. degree in computer science with a specialization in energy management in smart grid from the Communications Over Sensors (ComSens) Research Laboratory, COMSATS University Islamabad, Islamabad, Pakistan in 2017 under the supervision of Dr. Nadeem Javaid. She has authored more than 20 research publications in international journals and conferences. Her

References (78)

  • J. Lago et al.

    Forecasting day-ahead electricity prices in Europe: The importance of considering market integration

    Applied Energy

    (2018)
  • Y. Liu et al.

    Electricity load forecasting by an improved forecast engine for building level consumers

    Energy

    (2017)
  • Y. Lu et al.

    An improved RBF neural network for short-term load forecast in smart grids

  • A.A. Munshi et al.

    Big data framework for analytics in smart grids

    Electric Power Systems Research

    (2017)
  • E. Raviv et al.

    Forecasting day-ahead electricity prices: Utilizing hourly prices

    Energy Economics

    (2015)
  • M.Q. Raza et al.

    Demand forecast of PV integrated bioclimatic buildings using ensemble framework

    Applied Energy

    (2017)
  • A.I. Saleh et al.

    A data mining based load forecasting strategy for smart electrical grids

    Advanced Engineering Informatics

    (2016)
  • H. Shayeghi et al.

    Simultaneous day-ahead forecasting of electricity price and load in smart grids

    Energy Conversion and Management

    (2015)
  • N. Singh et al.

    Short term electricity price forecast based on environmentally adapted generalized neuron

    Energy

    (2017)
  • A. Tascikaraoglu et al.

    A review of combined approaches for prediction of short-term wind speed and power

    Renewable and Sustainable Energy Reviews

    (2014)
  • J. Wang et al.

    A novel model: Dynamic choice artificial neural network (DCANN) for an electricity price forecasting system

    Applied Soft Computing

    (2016)
  • L. Xiao et al.

    A combined model based on data pre-analysis and weight coefficients optimization for electrical load forecasting

    Energy

    (2015)
  • L. Zhang et al.

    A survey of randomized algorithms for training neural networks

    Information Sciences

    (2016)
  • M. Afshin et al.

    On efficient tuning of LS-SVM hyper-parameters in short-term load forecasting: A comparative study

  • A. Agarwal

    Introduction to artificial neural networks

    Medium

    (2019, December 11)
  • M. Alamaniotis et al.

    Genetic optimal regression of relevance vector machines for electricity pricing signal forecasting in smart grids

    IEEE Transactions on Smart Grid

    (2015)
  • V. Alto

    Neural networks: Parameters, hyperparameters and optimization strategies

    Medium: Towards Data Science

    (2019, July 5)
  • K. Amarasinghe et al.

    Deep neural networks for energy load forecasting

  • J.S. Bergstra et al.

    Algorithms for hyper-parameter optimization

    Advances in neural information processing systems

    (2011)
  • B.P. Bhattarai et al.

    Big data analytics in smart grids: State-of-the-art, challenges, opportunities, and future directions

    IET Smart Grid

    (2019)
  • F.M. Bianchi et al.

    Short-term electric load forecasting using echo state networks and PCA decomposition

    IEEE Access

    (2015)
  • A. Darwish et al.

    A survey of swarm and evolutionary computing approaches for deep learning

    Artificial Intelligence Review

    (2020)
  • X. Dong et al.

    Short-term load forecasting in smart grid: A combined CNN and K-means clustering approach

    2017 IEEE international conference on big data and smart computing (BigComp)

    (2017)
  • T. Elsken et al.

    Neural architecture search: A survey

    (2018)
  • A.T. Eseye et al.

    Short-term wind power forecasting using a double-stage hierarchical hybrid GA-ANN approach

  • A. Garulli et al.

    Models and techniques for electric load forecasting in the presence of demand response

    IEEE Transactions on Control Systems Technology

    (2015)
  • J.P. González et al.

    Forecasting functional time series with a new Hilbertian ARMAX model: Application to electricity price forecasting

    IEEE Transactions on Power Systems

    (2018)
  • S. Gupta et al.

    Support-vector-machine-based proactive cascade prediction in smart grid using probabilistic framework

    IEEE Transactions on Industrial Electronics

    (2015)
  • M.T. Hagan et al.

    Training feedforward networks with the Marquardt algorithm

    IEEE Transactions on Neural Networks

    (1994)
  • Cited by (101)

    View all citing articles on Scopus

    Rabiya Khalid received the MCS degree from Mirpur University of Science and Technology, Mirpur (Azad Kashmir), Pakistan, in 2014, and the M.S. degree in computer science with a specialization in energy management in smart grid from the Communications Over Sensors (ComSens) Research Laboratory, COMSATS University Islamabad, Islamabad, Pakistan in 2017 under the supervision of Dr. Nadeem Javaid. She has authored more than 20 research publications in international journals and conferences. Her research interests include data science and blockchain in smart/micro grids. Currently she is working as research associate and pursuing a PhD in the same lab and under the same supervision.

    Nadeem Javaid received the bachelor degree in computer science from Gomal University, Dera Ismail Khan, Pakistan, in 1995, the master degree in electronics from Quaid-i-Azam University, Islamabad, Pakistan, in 1999, and the Ph.D. degree from the University of Paris-Est, France, in 2010. He is currently an Associate Professor and the Founding Director of the Communications Over Sensors (ComSens) Research Laboratory, Department of Computer Science, COMSATS University Islamabad, Islamabad. He has supervised 120 master and 16 Ph.D. theses. He has authored over 900 articles in technical journals and international conferences. His research interests include energy optimization in smart/micro grids, wireless sensor networks, big data analytics in smart grids, and blockchain in WSNs, smart grids, etc. He was recipient of the Best University Teacher Award from the Higher Education Commission of Pakistan, in 2016, and the Research Productivity Award from the Pakistan Council for Science and Technology, in 2017. He is also associate editor of IEEE Access, editor of the International Journal of Space-Based and Situated Computing and editor of Sustainable Cities and Society.

    View full text