Engineering advanceA survey on hyperparameters optimization algorithms of forecasting models in smart grid
Introduction
The advancement in technology and increased usage of smart devices bring about the concept of big data. The production of data is growing rapidly and according to Munshi and Mohamed (2017), the volume of big data would increase by the factor of 300 in the upcoming years. The cost of data storage has also been reduced which is paving the way of storing more data and use it in the future. It has become a new focus for the researchers while playing a very important role in engineering, science and technology domains to acquire the efficient solution of the problems. In SG, a huge volume of data is being gathered and stored from sensors, smart meters and other smart devices which can be used for efficient planning and forecasting. So, forecasting using big data has become the new hot topic in this domain. The rapid increment in electricity consumption, intermittent nature of renewable energy sources (RESs) and fluctuations in demand are serious issues of a power system. The core aims of SG are to achieve the balance between demand and supply of electricity, increase the grid's reliability and efficiency and make grid environment friendly. Forecasting, in this regard, plays a very important role. It enables the utility to plan and organize the future decisions related to power generation, electricity price and coordination of electricity generating units and get maximum benefits out of them. Electricity load and price forecasting have gained great attention from researchers in this area as these two factors have a great influence on maintaining the stability of the grid. Additionally, forecasting of power failure, stability of transformers and power network, anomalies, blackouts and energy generation forecasting of RES are also studied and their solutions are provided in the literature.
The prediction of future values of SG components has a great importance as they play an important role as an input to the current decisions (Yusof & Mustaffa, 2016). For example, the knowledge of load demand in the future can play a very important role to set the electricity price values. It is also important for the utility to make different policies related to energy. Several decisions related to power, based on the information of future load. Similarly, a consumer can use the forecasted values of electricity prices and change its load consumption pattern accordingly. In recent years, researchers have proposed a large number of forecasting models for accurate load and price prediction. They can be used in the maintenance of power networks, better scheduling of energy generators, continuous energy provision to the consumers, achieve stability in demand and supply of electricity and maintain the grid reliability. Moreover, effective planning and decision making can save millions of dollars which are very important for the economical growth of a company as well as country.
In literature, researchers have proposed several forecasting methods to predict the load and price of electricity. Fig. 1 shows the classification of forecasting algorithms. Support vector machine (SVM) and neural networks (NNs) are commonly used forecasting algorithms by researchers to make predictions in the SG area. Different variants of these algorithms are also available and used. Bayesian networks (BN) are also one of the frequently used forecasting methods. In addition to these three types, several other forecasting techniques are also implemented for the prediction in SG. In forecasting algorithms, the accuracy is greatly affected by their hyperparameters. So, the values of these parameters should be chosen carefully.
In this paper, we have presented a survey of algorithms used for the optimization of hyperparameters in SG. Their values vary from problem to problem and need accurate optimized values for correct prediction. Inefficient optimization of these parameters results in poor accuracy results and a model that could be the best choice for a forecasting model, does not perform well. To overcome this issue, different optimization methods are applied to optimize these parameters. Fig. 1 shows the optimization methods commonly used for parameters optimization of forecasting models. Here, grid search, gradient descent and cross validation are the frequently used methods. Nature-inspired methods are also proposed by researchers to efficiently optimize these parameters. In the existing literature (Afshin, Sadeghian, & Raahemifar, 2007; Han, Jiang, Ling, & Su, 2019; Jiang et al., 2013; Yusof & Mustaffa, 2016; Zhang & Suganthan, 2016), similar surveys are presented. However, Yusof and Mustaffa (2016) and Afshin et al. (2007) surveyed the hyperparameter tuning methods of the least square support vector machine (LSSVM) only. Similarly, in Zhang and Suganthan (2016), randomized algorithms are surveyed for tuning NNs, Han et al. (2019) presents the survey of metaheuristic algorithms to train random single-hidden layer feedforward neural network (RSLFN), and Jiang et al. (2013) contains the survey of heuristic algorithms for tuning SVM. In our survey, we have reviewed both nature-inspired and statistical methods to tune the hyperparameters of SVM, NNS, BNs and their variants. The contributions of our survey are as follows:
- •
A detailed review of forecasting models (from 2014 and onwards) and optimization methods used to tune hyperparameters of these models is presented.
- •
Data preprocessing methods used in these studies are also discussed.
- •
All the forecasting models are critically analyzed and future research directions are also presented.
- •
In related work section, a survey of similar survey papers is presented and their recency score is also computed.
The rest of the paper is organized as follows: Section 2 contains similar work and in Section 3, we have discussed the proposed framework of a forecasting model that contains all the necessary steps for data forecasting, from data gathering to the final output phase. Section 4 contains a detailed discussion on forecasting techniques and optimizations methods used to optimize the values of their hyperparameters. In Section 5, common data preprocessing methods are discussed. Section 6 contains the critical analysis and findings of this survey. In Section 7, future directions related to hyperparameter optimizers are discussed. Finally, paper is concluded in Section 8.
Section snippets
Related work
Hyperparameter optimization is considered very important for the forecasting accuracy of algorithms. Researchers are using the already existing optimizers and also proposing new optimization algorithms for their tuning. Improvement of forecasting accuracy is an ongoing research area. So, to summarize this research and provide compact information to the readers, survey papers are published. The overview of some of the most related literature to our work is given below.
Forecasting model
Forecasting can be defined as a process in which current and past data values are analyzed to predict future values. In SG, electricity price and demand forecasting are of great importance. With the advancement in technology and increased Internet of things applications, a huge amount of data is gathered. This data is used to predict future energy demand, price, fault detection, electricity theft, etc. Researchers are actively working in this area and proposing efficient forecasting models with
Classification of tuning methods
In SG, the hyperparameters of forecasting algorithms are tuned using different optimizations techniques. The commonly used methods are grid search, cross validation, gradient descent and naive Bayes (NB). Nature-inspired heuristic algorithms are also getting popular in this field. Fig. 6 contains detailed classification of these algorithms. They are classified into two major categories: nature-inspired algorithms and other statistical methods.
Data preprocessing
In the previous section, we have discussed optimization methods for the hyperparameter tuning of forecasting algorithms in detail. Another important factor which affects the forecasting accuracy of these methods is the quality of data. A data set mostly includes noise in it, and it needs to be cleaned before using it for forecasting. In this section, we are going to discuss some data preprocessing methods, used in literature for data cleaning.
Wang, Xu, et al. (2017) have proposed a novel
Critical analysis
In this section, the analysis of the frequently used hyperparameter optimization methods for forecasting algorithms used in the SG domain is presented. Hyperparameter training is very important for the efficient forecasting. It trains the model according to the dataset and tuning these parameters improves the forecasting accuracy significantly. So, we have discussed the tuning methods used by researchers in recent years. These methods are compared in terms of their performance in optimization.
Future directions
The study of existing literature depicts the improvement in hyperparameter tuning is a continuous research area. Some future directions and challenges are identified from the available literature. In this section, we discuss some important future directions for hyperparameter optimizers for their improvement.
- 1.
The SG is evolving with each passing day and new actors are being integrated into it. So, it will require new models and applications based on forecasting algorithms e.g. ANN. These newly
Conclusion
This paper presents a brief and comprehensive survey of optimization techniques used for the optimization of hyperparameters of the forecasting model in SG. From literature, it is observed that the grid search and cross validation techniques are commonly used methods but as the size of the dataset increases, they require more computational time. On the other hand, researchers have applied nature-inspired heuristic optimization techniques to optimize these parameters. These techniques work
Conflict of interest
The authors declare that there is no conflict of interest.
Rabiya Khalid received the MCS degree from Mirpur University of Science and Technology, Mirpur (Azad Kashmir), Pakistan, in 2014, and the M.S. degree in computer science with a specialization in energy management in smart grid from the Communications Over Sensors (ComSens) Research Laboratory, COMSATS University Islamabad, Islamabad, Pakistan in 2017 under the supervision of Dr. Nadeem Javaid. She has authored more than 20 research publications in international journals and conferences. Her
References (78)
- et al.
A review on the selected applications of forecasting models in renewable power systems
Renewable and Sustainable Energy Reviews
(2019) - et al.
Towards efficient energy management in smart grids considering microgrids with day-ahead energy forecasting
Electric Power Systems Research
(2020) - et al.
Multiscale stochastic prediction of electricity demand in smart grids using Bayesian networks
Applied Energy
(2017) - et al.
Short-term electricity load forecasting of buildings in microgrids
Energy and Buildings
(2015) - et al.
Smart grid data analytics framework for increasing energy savings in residential buildings
Automation in Construction
(2016) - et al.
Energy forecasting for event venues: Big data and prediction accuracy
Energy and Buildings
(2016) - et al.
A survey on metaheuristic optimization for random single-hidden layer feedforward neural network
Neurocomputing
(2019) - et al.
A new hybrid modified firefly algorithm and support vector regression model for accurate short-term load forecasting
Expert Systems with Applications
(2014) - et al.
Extended forecast methods for day-ahead electricity spot prices applying artificial neural networks
Applied Energy
(2016) - et al.
Forecasting spot electricity prices: Deep learning approaches and empirical comparison of traditional algorithms
Applied Energy
(2018)
Forecasting day-ahead electricity prices in Europe: The importance of considering market integration
Applied Energy
Electricity load forecasting by an improved forecast engine for building level consumers
Energy
An improved RBF neural network for short-term load forecast in smart grids
Big data framework for analytics in smart grids
Electric Power Systems Research
Forecasting day-ahead electricity prices: Utilizing hourly prices
Energy Economics
Demand forecast of PV integrated bioclimatic buildings using ensemble framework
Applied Energy
A data mining based load forecasting strategy for smart electrical grids
Advanced Engineering Informatics
Simultaneous day-ahead forecasting of electricity price and load in smart grids
Energy Conversion and Management
Short term electricity price forecast based on environmentally adapted generalized neuron
Energy
A review of combined approaches for prediction of short-term wind speed and power
Renewable and Sustainable Energy Reviews
A novel model: Dynamic choice artificial neural network (DCANN) for an electricity price forecasting system
Applied Soft Computing
A combined model based on data pre-analysis and weight coefficients optimization for electrical load forecasting
Energy
A survey of randomized algorithms for training neural networks
Information Sciences
On efficient tuning of LS-SVM hyper-parameters in short-term load forecasting: A comparative study
Introduction to artificial neural networks
Medium
Genetic optimal regression of relevance vector machines for electricity pricing signal forecasting in smart grids
IEEE Transactions on Smart Grid
Neural networks: Parameters, hyperparameters and optimization strategies
Medium: Towards Data Science
Deep neural networks for energy load forecasting
Algorithms for hyper-parameter optimization
Advances in neural information processing systems
Big data analytics in smart grids: State-of-the-art, challenges, opportunities, and future directions
IET Smart Grid
Short-term electric load forecasting using echo state networks and PCA decomposition
IEEE Access
A survey of swarm and evolutionary computing approaches for deep learning
Artificial Intelligence Review
Short-term load forecasting in smart grid: A combined CNN and K-means clustering approach
2017 IEEE international conference on big data and smart computing (BigComp)
Neural architecture search: A survey
Short-term wind power forecasting using a double-stage hierarchical hybrid GA-ANN approach
Models and techniques for electric load forecasting in the presence of demand response
IEEE Transactions on Control Systems Technology
Forecasting functional time series with a new Hilbertian ARMAX model: Application to electricity price forecasting
IEEE Transactions on Power Systems
Support-vector-machine-based proactive cascade prediction in smart grid using probabilistic framework
IEEE Transactions on Industrial Electronics
Training feedforward networks with the Marquardt algorithm
IEEE Transactions on Neural Networks
Cited by (101)
Industrial kitchen appliance consumption forecasting: Hour-ahead and day-ahead perspectives with post-processing improvements
2024, Computers and Electrical EngineeringAn approach towards building robust neural networks models using multilayer perceptron through experimentation on different photovoltaic thermal systems
2023, Energy Conversion and ManagementResearch on the potential for China to achieve carbon neutrality: A hybrid prediction model integrated with elman neural network and sparrow search algorithm
2023, Journal of Environmental ManagementMachine learning for the management of biochar yield and properties of biomass sources for sustainable energy
2024, Biofuels, Bioproducts and Biorefining
Rabiya Khalid received the MCS degree from Mirpur University of Science and Technology, Mirpur (Azad Kashmir), Pakistan, in 2014, and the M.S. degree in computer science with a specialization in energy management in smart grid from the Communications Over Sensors (ComSens) Research Laboratory, COMSATS University Islamabad, Islamabad, Pakistan in 2017 under the supervision of Dr. Nadeem Javaid. She has authored more than 20 research publications in international journals and conferences. Her research interests include data science and blockchain in smart/micro grids. Currently she is working as research associate and pursuing a PhD in the same lab and under the same supervision.
Nadeem Javaid received the bachelor degree in computer science from Gomal University, Dera Ismail Khan, Pakistan, in 1995, the master degree in electronics from Quaid-i-Azam University, Islamabad, Pakistan, in 1999, and the Ph.D. degree from the University of Paris-Est, France, in 2010. He is currently an Associate Professor and the Founding Director of the Communications Over Sensors (ComSens) Research Laboratory, Department of Computer Science, COMSATS University Islamabad, Islamabad. He has supervised 120 master and 16 Ph.D. theses. He has authored over 900 articles in technical journals and international conferences. His research interests include energy optimization in smart/micro grids, wireless sensor networks, big data analytics in smart grids, and blockchain in WSNs, smart grids, etc. He was recipient of the Best University Teacher Award from the Higher Education Commission of Pakistan, in 2016, and the Research Productivity Award from the Pakistan Council for Science and Technology, in 2017. He is also associate editor of IEEE Access, editor of the International Journal of Space-Based and Situated Computing and editor of Sustainable Cities and Society.