An LSTM-based neural network method of particulate pollution forecast in China

We obtain the national urban pollution data from the national urban air quality real-time release platform (http://106.37.208.233:20035/, last access: 22 November 2020). The meteorological data is from the China surface climatic data daily data set V3.0, which is from the China Meteorological Data Network (https://data.cma.cn, last access: 22 November 2020). We select the national urban pollution data and meteorological data of 359 cities from period 2015 to 2017. We have 1095 data samples for each city in our data set and our LSTM model is based on the time dimension. The details of the data are summarized in table S1 (available online at stacks.iop.org/ERL/16/044006/mmedia). The description of data preprocessing and the temporal feature are in supplemental text S1. The combined data sets for our research are saved at figshare (Zhu 2021).

Data collection sites are mainly distributed in the eastern region of China, especially the Yangtze River Delta, the Pearl River Delta and the Beijing–Tianjin–Hebei region, with relatively few sites in the western region. Therefore, in this paper, we select a few representative cities (Beijing, Taiyuan, Shanghai, Nanjing and Guangzhou) from different geographic regions (Jing–Jin–Ji, Northwest China, Yangtze River Delta and Pearl River Delta) of China and cities surrounding them within a certain distance (e.g. 100 km, 200 km) for this study (figure 1(a)). Due to the geographical proximity between Beijing and Taiyuan as well as Nanjing and Shanghai, some of the selected cities within 200 km are with overlapped areas (the red circle in figure 1(a)).

Zoom In Zoom Out Reset image size

Long short-term memory (LSTM) neural network is a widely used RNN architecture in the field of deep learning. It has a tree-like hierarchical structure, consisting of input layer, hidden layer and output layer (figure 1(b)), while its network nodes are recursive with the input information connected to each other in sequential order. Each hidden layer contains LSTM structure and dropout layer. LSTM structure is the core part of the entire neural network. Each LSTM neuron contains three control gates: input gate, forgetting gate and output gate. The input gate learns to decide when to let the activation signal into the cell, the output gate learns when to let the activation signal out of the cell, and the forgetting gate learns when to let the cell of the last moment into the cell of the next moment. In the forgetting gate, only the information conforming to algorithm authentication will remain, while the information not conforming will be forgotten, thus it effectively solves the problem of long-term dependence (Greff et al 2017). In the dropout layer, neural network cells are temporarily discarded from the network with a certain probability during the training process, which is a means to prevent overfitting and has good fault tolerance.

In our model, the prediction object is the target pollutant concentration of a central city, and the prediction factors are the historical daily data of meteorological elements and pollution elements of the central city and its surrounding n_SITE stations in the past r days. Specifically, the target pollutant is daily PM₁₀ in this study. The input factors include d factors such as daily average temperature, daily average humidity, maximum wind speed, maximum wind direction, SO₂ concentration, NO₂ concentration, CO concentration, O₃ concentration for 8 h, PM_2.5 concentration and PM₁₀ concentration (table 1).

We constructed the input factor matrix W, and its dimension is r × n_SITE × d (figure 1(c)). One or more LSTM cells constitute the middle layer of the prediction model, and the number of layers is denoted as n_LAYER . Each LSTM layer contains n_NEU neurons and is followed by a dropout layer. The output layer is where we find the predicted target p_PRE . The forecaster is denoted as p_PRE while the corresponding observed value is denoted as p_OBS . After e times of prediction, the prediction sequence p_PRE would be achieved. The R-square and root mean square error (RMSE) of p_PRE and p_OBS are calculated to evaluate the prediction performance of different model configurations.

During the training process, the optimization function used in the LSTM model is the 'Adam' algorithm (Zhang et al 2019) and the loss function is 'Mean_Squared_Error'.

In general, the prediction effect of the model is related to the parameters of r, n_SITE, n_LAYER and n_NEU . In this study, we used Python and its deep learning libraries Tensorflow and Keras to build and implement the above neural network algorithms. We set up a control experiment of multiple parameter configurations, and then discuss the capabilities of our LSTM model in different regions of China.

We divided the data set with time length of L days into two sets, with 70% of the available data allocated as training set and the remaining 30% data as test set. A summary of experiment settings is shown in table S3. We adopted the control variables for the selected representative cities (Beijing, Taiyuan, Shanghai, Nanjing, and Guangzhou) with only one changeable variable in each group of experiments. Finally, we used R² and RMSE to evaluate the experimental results to explore the influence of parameters such as the number of layers in LSTM (supplemental text S2), the scope of surrounding cities (supplemental text S3), the days of input data as well as different regions that the model applied to (supplemental text S4). Based on the experimental results, we determined the best model is double-layer LSTM with previous one observed data as input within a certain geographical range (DLP1–LSTM model; DLP1, double-layer with previous ONE observed data).

To evaluate superiority of the LSTM model in PM₁₀ prediction in this paper, we compared it with five other methods including traditional statistical methods and machine learning models: ordinary least squares regression (OLSR), Bayesian ridge regression, support vector regression (SVR), multi-layer perceptron (MLP), and Random Forest Regression. We will briefly introduce these methods in the following paragraphs.

OLSR, which seeks the best match for data by minimizing L2 norm errors, is a mathematical optimization technique (Moutinho and Hutcheson 2011). In this algorithm, OLSR is used to fit a linear model using coefficients in Linear Regression to minimize the sum of squares of differences between the actual observed data and the predicted data.

Bayesian Ridge Regression solves these regression problems by imposing a penalty on the Regression coefficients on the basis of OLSR. In other words, a regular term is added to the deviated sum of squares function, which is used to adjust parameters and delete those related terms (Massaoudi et al 2020).

SVR is different from traditional regression methods. For SVR, as long as the predication f(x) does not deviate too much from Y, the prediction can be regarded as correct and there is no need to calculate the error. SVR is still a convex optimization problem and thus ensures the global solution give it nonlinear prediction ability and generalization performance (Uçak and Günel 2016).

The creation of feedforward neural network of MLP starts from the most basic form of a single perceptron. A sensor has one or more inputs, biases, activation functions, and a single output. The sensor takes the inputs, multiplies them by certain weights, and passes them to the activation function to produce the output. The input layer of an MLP consists of all the variables in the individual neurons, and the output layer consists of response variables. The input layer and the hidden layer include a constant neuron associated with the intercepting synapse or deviation (that is, a synapse not directly affected by any covariable called interception) (Egbo 2018).

Random Forest Regressor is an ensemble method by creating forest with multiple decision trees during training. Each tree contains an arrangement of decision nodes, based on which the tree is divided into different branches until the termination point (leaf) is reached. Each decision node depends on whether the value of the input feature exceeds a certain threshold. The final prediction is obtained by averaging each tree prediction that provided by training it independently (Pillai et al 2020).

In this paper, the modules in the Scikit-earn library are used to implement the models for comparison. To ensure reliability of the comparison results, the data preprocessing and the input factors in other machine learning methods are kept consistent with our LSTM model.

In order to study the applicability of double-layer LSTM model to PM₁₀ prediction in different regions of China, we applied our DLP1–LSTM model to several typical metropolitans that represent different features of pollution in China. They are Beijing, Taiyuan, Shanghai, Nanjing and Guangzhou. We set up the configuration of using observation data from cities within 100 km of circle around of the target city and 1 d before the prediction day. The training convergence curves (figure S2) show that the DLP1–LSTM prediction model gradually converges for all five target cities. This further reflects the good applicability of the prediction model in China.

Figure 2 is the time series diagram of the predictions and observations of all five cities. In general, the predictions match the trends of observations of all five cities. The predictions could match most of the peaks and valleys. We also know from figure 2 that the observed PM₁₀ of Guangzhou is relatively lower than that of other cities and has fewer sharp changes. Figure S3 shows that in the representative cities, the predicted values of the DLP1–LSTM is roughly positively correlated with the observations, and the slope of the fitting curve is close to 1. In addition, Nanjing has the highest R² (0.828), while Shanghai has the lowest R² (0.642) among these five cities. The predictions of PM₁₀ concentrations in these cities all have good correlations (R = 0.81–0.91) with observations. Guangzhou has the smallest RMSE, while Taiyuan has the largest RMSE.

Zoom In Zoom Out Reset image size

In summary, our configuration of double LSTM model is able to make accurate predictions of daily PM₁₀ concentrations for various city metropolitans within a wide range of pollution levels in China.

To further explore the predictive power of the DLP1–LSTM model, we analyzed the sensitivities of the model to different levels of pollution and also its performance in the five representative cities of different regions. We divided the pollution into three levels: mild (0–50 μg m⁻³), moderate (50–100 μg m⁻³) and polluted (>100 μg m⁻³), and then tested the prediction accuracy of our model on next-day trend (increase or decrease from the previous day) for different pollution levels. Overall, the model has the best trend prediction capability for PM₁₀ concentration in heavy pollution situation (table S7).

Figure S4 compares the next-day changes of predictions to observations. Positive value means the PM₁₀ concentration increases when compared to the previous day, while negative value means it decreases when compared to the previous day. Beijing has more days with moderate and heavy pollution, Nanjing has more days with moderate and mild pollution, and Taiyuan has more days with heavy pollution. The prediction accuracy of the next-day trend is much better for polluted cases of these three cities. However, Guangzhou has the most days of mild pollution, and the corresponding correlation of prediction of mild pollution is much better than other cities (figure S3 and table 2). Shanghai has the lowest accuracy of the next-day trend forecast, at about 70%. The prediction ability of the model is weak for mild (R² = 0.12) and moderate (R² = 0.17) pollution days.

It can be seen from figure S3 and table 2 that the prediction ability of next-day trend in Shanghai is weaker than that in the other cities. It is due to the large coastal area of Shanghai, and its landform is mostly plain, which is conducive to the diffusion of pollutants. In addition, there is a lot of rain in summer, especially in the Meiyu season. The relative humidity is high and is negatively correlated with the concentration of PM₁₀ (Jian et al 2019), that is, it is conducive to the settlement of PM₁₀. Moreover, Shanghai has less pollution in general and fewer days of heavy pollution, which reduces the ability of the model to capture its characteristics. Guangzhou, which also faces the sea and has a relatively lower degree of pollution, is special with more hills and high in the northeast and low in the southwest. As a result, the terrain of Guangzhou is not conducive to the diffusion of pollutants and the DLP1–LSTM model is able to capture the data features well.

In general, the prediction accuracy of increasing or decreasing of the next day is about 70%–80% when the PM₁₀ concentration of the current day is known, and it varies slightly among different representative cities. The model is more suitable for predicting heavy pollution events (PM₁₀ concentration >100 μg m⁻³) with some regional differences.