Probability density forecasting of wind power based on multi-core parallel quantile regression neural network

Highlights

• This is a parallel quantile regression neural network wind power probability density forecasting model.

• The algorithm can improve the efficiency of quantile regression neural network.

• Results is evaluation by metrics of Speedup and Parallel efficiency.

Abstract

The large-scale utilization of wind energy brings severe challenges to the dispatching operation of power systems. Currently, the probability density prediction method combining quantile regression neural network (QRNN) with Epanechnikov kernel function is an excellent algorithm for wind power prediction, which can give the comprehensive probability distribution of future wind power and effectively quantify the uncertainty of wind power generation. However, existing probability density prediction methods process data sequentially in different quantiles, and computational time costs multiply with the increase of training data. It affects the practicality of the probability density prediction method. To overcome this issue, this paper proposes a multi-core parallel quantile regression neural network (MPQRNN) based on parallel master–slave (MS) model. The algorithm divides the complex prediction tasks at all quantiles into multiple parallel sub-tasks, which are independently run on different cores, so that performance advantages of the multi-core CPU can be fully utilized for improving the computational efficiency of the joint operation model. We compare four different scale sample sets under different process numbers. The influence of different CPU core numbers on the parallel performance of MPQRNN are analyzed by the algorithmic nature of speedup and parallel efficiency. To demonstrate the effectiveness of the proposed model, comparative experiments of other four traditional models are carried out on data sets. The simulation results demonstrate that the MPQRNN can not only improve the training efficiency of QRNN, but also obtain precise results of wind power forecasting, showing potential value and utility for complex power system.

Introduction

In the last few years, due to the increase in thermal pollution and greenhouse gases caused by traditional fossil fuel power generation, renewable energy has become an epidemic choice for power generation and a key component of government development strategies [1]. Among renewable energy sources, wind and solar power generation are becoming more considerable because of their universality and relatively low production costs [2]. Wind energy is a clean, pollution-free renewable energy source with huge reserves and has the potential to produce energy in a successive and sustainable way in renewable energy. It has become one of the most popular renewable energy sources in the world due to its wide availability [3]. According to the report, due to the drastic reduction in the cost of wind, solar and energy storage technologies, nearly half of the world’s electricity will be supplied by these two rapidly developing renewable energy sources by 2050 [4]. Without a shadow of doubt, wind power is expected to maintain rapid growth for a long time to come. However, one of the main obstacles to wind power generation is that the uncertainty of the wind regeneration process gives rise to the unschedulability and randomness of wind plant production capacity, which poses a huge challenge to grid dispatching and power trading [5]. Besides, the experience of wind power operation in various countries shows that large-scale wind power access may touch off a series of safety and stability problems. Comprehensive and accurate wind power forecasting is conducive to early scheduling, reducing wind power uncertainty, optimizing unit commitment and load scheduling. It is an indispensable tool to improve the stability of the power system and the economic benefits of wind power generation [6].

In recent years, many researches focusing on the wind power prediction models have been carried out, such as back-propagation neural network (BPNN) [7], fuzzy neural network (FNN) [8], wavelet analysis [9], support vector machine (SVM) [10] and regression analysis [11], providing strong support for wind power prediction. Among them, BPNN has strong nonlinear mapping and generalization ability, but it is prone to produce slow convergence and local optimization; FNN has great advantages in dealing with nonlinear and fuzzy problems because of the organic combination of fuzzy model [12], [13] and neural network, but the convergence speed of the model is slow and the ability of global approximation is poor; wavelet analysis has good approximation and fault-tolerance capacity, but the choice of wavelet basis function and parameter initialization are not based on certain criteria; SVM can eliminate plenty of redundant samples with good robustness, but it is tough to train large-scale samples and solve multi-classification problems; regression analysis method has high training speed and small error rate when data is small, but it is not available for processing mass of data. These wind power prediction algorithms cannot meet the increasingly intricate wind power prediction requirements owing to the limitation of accuracy and scope of application.

The above models place emphasis on deterministic spot wind power predictions. However, they are unable to reveal the indeterminacy characteristics of wind power. On the contrary, the probabilistic prediction of wind power has shined in recent years, because it has the ability to provide accurate deterministic prediction, quantify uncertainty of wind power output, and construct a continuous probability density curve of predicted wind power [14]. This is more significant for power companies and system operators than point predictions. In the researches of probability prediction of wind power, quantile regression (QR) is one of the prominent regression analysis methods frequently used in statistics and econometrics territories. Without pondering the distribution pattern of random variables, QR can obtain the overall distribution characteristics of the response variable by a specified appropriate regression function, which is sturdier when handling outliers in the interpreted measurements [15]. Therefore, QR has been extensively applied in wind power forecasting [16]. However, straightforward linear QR is arduous to deal with complex nonlinear problems [17]. Therefore, in order to exhaustively bring the nonlinear relationship between the dependent variable and the independent variable to light, Taylor [18] first proposed the QRNN model, which is a nonparametric composition method of artificial neural network (ANN) [19] and QR. The nonlinear method combines the advantages of both. Pradeepkumar et al. [20] forecasted the volatility of financial time series by using particle swarm optimization algorithm to optimize trained QRNN. Cannon [15] demonstrated potential superiorities of QRNN model for precipitation downscaling task. He and Li. [16] utilized a hybrid method based on quantile regression neural network and Epanechnikov kernel function using unbiased cross-validation (QRNNE-UCV) to accomplish the probability density prediction, which validly characterized the indefiniteness of wind power generation. These empirical researches further demonstrate the meliority of the QRNN approach. QRNN not only demonstrates the entire conditional distribution of the explained variable, but also handles sophisticated nonlinear problems. Applying the QRNN method to wind power prediction, specific conditional quantiles of the wind power under different quantiles can be acquired, with better prediction veracity.

However, it has a key drawback. The aggregation operation mechanism of QRNN is complicated, and the computational complexity of its training is increased exponentially by the traditional neural network structure. Furthermore, the integration of informatization and industrialization in the power industry has promoted the rapid growth of wind power data, inevitably making wind power forecasting increasingly complicated and posing a significant challenge to the calculation efficiency of the model [21]. The QRNN algorithm mentioned above can provide precise wind power predictions, but the computation time cost increases dramatically with the increase of the data size and model complexity. In particular, when the scale of a wind power system reaches a certain level, the calculation time cost is absolutely intolerable. Although many researchers have optimized the QRNN model with outstanding results in some aspects over the years, there are still some imperfections such as slow training speed. These improved algorithms still cannot meet the requirements of practical application. In general, there exist two basic ways to increase computational efficiency. One is to improve the classical algorithm or to seek new and better algorithms. The other is to use new computer technologies, including hardware and software. Therefore, a great deal of researches has been devoted to the efficient utilization of high performance computing (HPC) platforms to enhance the operation efficiency of models [22]. This article will target the latter, and our focus is on the parallelization of QRNN method, especially how to get the utmost out of the current epidemic multi-core resources.

Multi-core processors are also known as chip multi-processor (CMP), or single-chip multiprocessors [23]. Multi-core processor refers to the incorporation of two or more intact computing engines (cores) in a single processor, each supporting a separate thread. In 1996, Stanford University proposed the idea of CMP for the first time. Then in 2001, IBM introduced the first commercial multi-core processor POWER4, followed by the large-scale application of Intel and AMD multi-core processor in 2005 [24]. Until now, multi-core processor has experienced more than a decade of development and become the mainstream of the market. In this process, the application range of multi-core processor has covered many domains such as multimedia computing, embedded device, personal computer, commercial server and high-performance computer.

Besides, the increasing popularity of multi-core processor provides the indispensable hardware foundation for computing tasks using parallel mechanisms. High-performance parallel computing technology has attracted increasing attention internationally, and gradually become a crucial pathway to improve computing efficiency [25]. It has been extensively utilized in scientific research and engineering technology, such as climate forecasting, seismic analysis and other complicated issues [26]. Parallel computing in a multi-core environment means decomposing the entire traditional serial processing task into multiple subtasks, and then allocating the subtasks to multiple different cores for processing [27]. In these cores, subtasks can be executed independently to speed up the calculation process. The main purpose of parallel computing: one is to provide faster computing speed than traditional computer; the other is to solve large-scale problems that traditional computers are hard to solve. Compared with traditional serial computing, multi-core can achieve parallel computing on ordinary machines, saving plenty of hardware resources and network resources and improving computational efficiency. Therefore, taking full advantage of multi-core parallel computing is a vital means to solve the serial processing speed bottleneck and realizes intricate, high-dimensional, computation-intensive tasks. Today, increasing computational efficiency through parallelization has been proven successfully in many domains. In [28], a new parallel artificial bee colony algorithm has been proposed for solving the vehicle routing problem effectively. Cheng et al. [24] presented a fine-grained parallel discrete differential dynamic programming (PDDDP) algorithm based on the Fork/Join parallel framework in multi-core environment to improve the computational efficiency for long-term operation of multi-reservoir hydropower systems. In [29], Bryan used HPC techniques for comprehensive space–time assessment and modeling of intricate social–ecological systems. In [30], high resolution modeling of wheat production, soil carbon and nitrogen dynamics in growing areas of Australia was carried out by combining parallel processing and grid computing. Huo et al. [31] proposed a multi-core parallel artificial swarm optimization (MPABC) algorithm based on a parallel hierarchical model and Fork/Join framework to optimize the hydrological model of Xinanjiang. To solve the efficiency requirement of progressive optimization algorithm (POA) in solving hydropower scheduling problems, a new efficient parallel progressive optimization algorithm is presented in [32]. Sha et al. [33] proposed a new deep parallel residual network (DPRN), which is an efficient algorithm to solve the problem of single image super-resolution. Madsen et al. [34] proposed a new method to parallelize Bayesian network structure learning, which can significantly improve the time performance. Masoodd et al. [35] proposed a parallel algorithm based on GPU to calculate alpha complexes, and the computational experiments on several biomolecules have proved its superior performance. Therefore, the study of parallel algorithms is also a matter of cardinal significance to solve the time-consuming problems of large wind power systems.

This paper presents the parallel master–slave model of QRNN (hereinafter referred to as MPQRNN). The parallel QRNN algorithm based on sub-point prediction task division mainly adopts master–slave (MS) parallel mode, and simultaneously combines the structure characteristics and operation mechanism of QRNN to design. Although the parallel MS framework is not the most efficient technique for good parallel performance, it has distinct advantages for attractive choices. First, the MS model can decompose the task recursively into many subtasks using a divide-and-conquer strategy, which is applicable for computationally intensive tasks. Second, the parallel MS framework can make the most of multi-core resources, making it pervasive application in computers with multiple cores. Finally, the MS model is one of the most effective and easiest-to-implement parallel computing methods, so it is much easier to design parallel algorithms for a given problem. Jin [36] utilized the MS model combined with a particle swarm optimization (PSO) algorithm to optimize antenna design. In [37], the researchers detailed a MS parallelization of PSO algorithm to solve a medium-scale biomechanical system identification problem. In [38], a parallel MS model of the cooperative PSO method was introduced, based on the decomposition of the original search space in subspaces of smaller dimension. Tang et al. [39] applied the MS and multi-population parallelization programs of the Epsilon-Nondominated Sorted Genetic Algorithm-II ($\epsilon -NSGAII$) on the water resources problems. Li et al. [40] proposed a MS parallel dynamic programming model on a HPC system, and applied the knowledge-based method to determine the power generation of the Three Gorges Project and the Gezhouba Project cascade hydropower plant in China. Zhao et al. [41] proposed the deep belief networks parallel computing method, and utilized it to predict traffic flow. Although MS parallel computing has been extensively used in other domain models, it has not been entirely explored in the territory of wind power forecasting operations, and few papers in the technical literature consider applying parallel algorithm to the territory of wind power prediction.

In summary, the parallelization of QRNN algorithm is still in its infancy, and the application of MS parallel framework in the optimization of wind power prediction model is relatively rare. To the best of our knowledge, this is the first parallel implementation based on QRNN. The feasibility and utility of the presented algorithm are tested by historical wind power data from Ontario of Canada. Case studies show that using MPQRNN method, users can easily achieve higher parallel efficiency and fully meet the precision of the original QRNN prediction. Compared with authors’ recent papers [16], [42], [43], this paper focuses on the efficiency of wind power probability density prediction. The MPQRNN model is established by combining the MS parallel model and QRNN. Our previous work in the field of probability density prediction was carried out in a single core environment, which wasted computer resources and could not handle too large a data set. This work effectively solved these problems. The case study results imply that parallel processing is a viable approach to improve the computation efficiency for wind power prediction and it is a very useful technique in future, especially with the rapid growth and scale of wind power system and the ever increasing level of computer hardware and software.

The main contributions of this article are as follows: (1) Considering the calculation time cost of QRNN, the MPQRNN method using multi-core resources is proposed, which can utilize the characteristics of multiple CPUs to simultaneously run the neural network training tasks under all the quantile points and relieve computational pressure. (2) In order to study the influence of CPU core number and training sample set size on MPQRNN model, this paper evaluates the wind power prediction results of four different scale data sets under different process numbers and compares them from the perspective of speedup and parallel efficiency. The results show that the operational performance is affected by the number of processes and the size of the sample. (3) Finally, the performances of serial and parallel QRNN models are verified through four cases in Ontario of Canada from three aspects: parallel performance, point prediction accuracy, and interval prediction indeterminacy. Meanwhile, the point prediction results of serial and parallel QRNN models are compared with those of QR probability density prediction, Takagi–Sugeno fuzzy neural network (TS-FNN), BPNN and radial basis function neural network (RBFNN). The case study shows that MPQRNN has higher computing speed, so as to improve the computational efficiency. With the expansion of the number of CPU cores, the advantage of parallel processing method will be more significant. But the speedup will decrease after reaching the peak value.

The rest of the structure of this paper is organized as follows: Section 2 introduces the relevant theories used in this paper, such as QRNN model, parallel master–slave framework, design integration of MPQRNN algorithm and probability density prediction method based on MPQRNN. Section 3 introduces the main indicators for evaluating parallel performance and forecasting performance of algorithms. Section 4 provides the experimental environment, data analysis and model application of wind power from Ontario, Canada, and a comparison with other traditional methods in the same scenario. The validity of the parallel algorithm is verified by the actual data. Finally, Section 5 summarizes the conclusions. The nomenclatures used in the study are introduced in Table 1.