Abstract
PM2.5 forecasting is an important scientific way to control environmental pollution and keep people healthy. To achieve high-performance PM2.5 forecasting, a new ensemble reinforcement learning gated unit model is presented in this study. The complete framework of this model mainly includes the following steps: In step I, the WPD method is applied to decompose PM2.5 data into 8 sub-series with different frequency types. In step II, the SAE-GRU method is presented to finish the establishment of sub-series forecasting model. Among them, SAE is used to obtain low-latitude features of PM2.5 data, and GRU is applied to finish PM2.5 sub-series forecasting. In step III, Q-learning is used to combine the every PM2.5 sub-series to get the final model prediction results. By comparing and analyzing the final results of all case study, it can be summarized that (1) Q-learning-based ensemble model integrates the subseries with different frequency types perfectly, and results prove that it is better than heuristic algorithm, and (2) the proposed ensemble reinforcement learning gated unit model can get prediction results beyond seventeen alternative models which include three most state-of-the-art models in all cases.
Similar content being viewed by others
References
Adelia R, Suyanto S, Wisesty UN (2019) Indonesian abstractive text summarization using bidirectional gated recurrent unit. Procedia Comput Sci 157:581–588. https://doi.org/10.1016/j.procs.2019.09.017
Adem K, Kiliçarslan S, Cömert O (2019) Classification and diagnosis of cervical cancer with stacked autoencoder and softmax classification. Expert Syst Appl 115:557–564. https://doi.org/10.1016/j.eswa.2018.08.050
Bai Y, Zeng B, Li C, Zhang J (2019) An ensemble long short-term memory neural network for hourly PM2.5 concentration forecasting. Chemosphere 222:286–294
Becerra-Rico J, Aceves-Fernández MA, Esquivel-Escalante K, Pedraza-Ortega JC (2020) Airborne particle pollution predictive model using Gated Recurrent Unit (GRU) deep neural networks. Earth Science Informatics:1–14
Biancofiore F, Busilacchio M, Verdecchia M, Tomassetti B, Aruffo E, Bianco S, Di Tommaso S, Colangeli C, Rosatelli G, Di Carlo P (2017) Recursive neural network model for analysis and forecast of PM10 and PM2. 5. Atmospheric Pollution Research 8(4):652–659
Błaszczyk E, Rogula-Kozłowska W, Klejnowski K, Kubiesa P, Fulara I, Mielżyńska-Švach D (2017) Indoor air quality in urban and rural kindergartens: short-term studies in Silesia, Poland. Air Qual Atmos Health 10(10):1207–1220
Cheng Y, Zhang H, Liu Z, Chen L, Wang P (2019) Hybrid algorithm for short-term forecasting of PM2. 5 in China. Atmos Environ 200:264–279
Ding Y, Ma L, Ma J, Suo M, Tao L, Cheng Y, Lu C (2019) Intelligent fault diagnosis for rotating machinery using deep Q-network based health state classification: a deep reinforcement learning approach. Adv Eng Inform 42:100977. https://doi.org/10.1016/j.aei.2019.100977
Doreswamy KSH, Km Y, Gad I (2020) Forecasting air pollution particulate matter (PM2.5) using machine learning regression models. Procedia Comput Sci 171:2057–2066. https://doi.org/10.1016/j.procs.2020.04.221
Hähnel P, Mareček J, Monteil J, O'Donncha F (2020) Using deep learning to extend the range of air pollution monitoring and forecasting. J Comput Phys 408:109278. https://doi.org/10.1016/j.jcp.2020.109278
Haidong S, Junsheng C, Hongkai J, Yu Y, Zhantao W (2020) Enhanced deep gated recurrent unit and complex wavelet packet energy moment entropy for early fault prognosis of bearing. Knowl-Based Syst 188:105022. https://doi.org/10.1016/j.knosys.2019.105022
Jiang P, Li C, Li R, Yang H (2019) An innovative hybrid air pollution early-warning system based on pollutants forecasting and Extenics evaluation. Knowl-Based Syst 164:174–192. https://doi.org/10.1016/j.knosys.2018.10.036
Kristjanpoller W, Minutolo MC (2015) Gold price volatility: a forecasting approach using the artificial neural network–GARCH model. Expert Syst Appl 42(20):7245–7251. https://doi.org/10.1016/j.eswa.2015.04.058
Kroon M, Whiteson S (2009) Automatic feature selection for model-based reinforcement learning in factored MDPs. In: International Conference on Machine Learning and Applications, ICMLA 2009, Miami Beach, Florida, USA, December 13-15, 2009
Liu H, Duan Z, Chen C (2019a) A hybrid framework for forecasting PM2. 5 concentrations using multi-step deterministic and probabilistic strategy. Air Qual Atmos Health 12(7):785–795
Liu H, Duan Z, Wu H, Li Y, Dong S (2019b) Wind speed forecasting models based on data decomposition, feature selection and group method of data handling network. Measurement 148:106971
Liu H, Jin K, Duan Z (2019c) Air PM2. 5 concentration multi-step forecasting using a new hybrid modeling method: comparing cases for four cities in China. Atmos Pollut Res 10(5):1588–1600
Liu H, Xu Y, Chen C (2019d) Improved pollution forecasting hybrid algorithms based on the ensemble method. Appl Math Model 73:473–486. https://doi.org/10.1016/j.apm.2019.04.032
Liu H, Duan Z, Chen C (2020a) A hybrid multi-resolution multi-objective ensemble model and its application for forecasting of daily PM2.5 concentrations. Inf Sci 516:266–292. https://doi.org/10.1016/j.ins.2019.12.054
Liu H, Long Z, Duan Z, Shi H (2020b) A new model using multiple feature clustering and neural networks for forecasting hourly PM2. 5 concentrations, and its applications in China. Engineering
Liu H, Yu C, Wu H, Duan Z, Yan G (2020c) A new hybrid ensemble deep reinforcement learning model for wind speed short term forecasting. Energy 202:117794. https://doi.org/10.1016/j.energy.2020.117794
Liu H, Yu C, Yu C, Chen C, Wu H (2020d) A novel axle temperature forecasting method based on decomposition, reinforcement learning optimization and neural network. Adv Eng Inform 44:101089. https://doi.org/10.1016/j.aei.2020.101089
Mirza AH, Kerpicci M, Kozat SS (2020) Efficient online learning with improved LSTM neural networks. Digit Signal Process 102:102742. https://doi.org/10.1016/j.dsp.2020.102742
Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533
Momennejad I, Russek EM, Cheong JH, Botvinick MM, Daw ND, Gershman SJ (2017) The successor representation in human reinforcement learning. Nat Hum Behav 1(9):680–692
Ocak H, Loparo KA, Discenzo FM (2007) Online tracking of bearing wear using wavelet packet decomposition and probabilistic modeling: a method for bearing prognostics. J Sound Vib 302(4–5):951–961
Peduzzi E, Pisoni E, Clappier A, Thunis P (2018) Multi-level policies for air quality: implications of national and sub-national emission reductions on population exposure. Air Qual Atmos Health 11(9):1121–1135
Sharma N, Taneja S, Sagar V, Bhatt A (2018) Forecasting air pollution load in Delhi using data analysis tools. Procedia Comput Sci 132:1077–1085. https://doi.org/10.1016/j.procs.2018.05.023
Solaiman T, Coulibaly P, Kanaroglou P (2008) Ground-level ozone forecasting using data-driven methods. Air Qual Atmos Health 1(4):179–193
Song J, Wang J, Lu H (2018) A novel combined model based on advanced optimization algorithm for short-term wind speed forecasting. Appl Energy 215:643–658
Xiong R, Cao J, Yu Q (2018) Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle. Appl Energy 211:538–548. https://doi.org/10.1016/j.apenergy.2017.11.072
Xu Y, Liu H (2020) Spatial ensemble prediction of hourly PM2. 5 concentrations around Beijing railway station in China. Air Qual Atmos Health:1–11
Xu F, Yang F, Fan X, Huang Z, Tsui KL (2020a) Extracting degradation trends for roller bearings by using a moving-average stacked auto-encoder and a novel exponential function. Measurement 152:107371. https://doi.org/10.1016/j.measurement.2019.107371
Xu Y, Liu H, Duan Z (2020b) A novel hybrid model for multi-step daily AQI forecasting driven by air pollution big data. Air Qual Atmos Health 13(2):197–207
Yerramilli A, Dodla VBR, Challa VS, Myles L, Pendergrass WR, Vogel CA, Dasari HP, Tuluri F, Baham JM, Hughes RL (2012) An integrated WRF/HYSPLIT modeling approach for the assessment of PM 2.5 source regions over the Mississippi Gulf Coast region. Air Qual Atmos Health 5(4):401–412
Yin P-Y, Chao C-H (2018) Automatic selection of fittest energy demand predictors based on cyber swarm optimization and reinforcement learning. Appl Soft Comput 71:152–164. https://doi.org/10.1016/j.asoc.2018.06.042
Zabalza J, Ren J, Zheng J, Zhao H, Qing C, Yang Z, Du P, Marshall S (2016) Novel segmented stacked autoencoder for effective dimensionality reduction and feature extraction in hyperspectral imaging. Neurocomputing 185:1–10
Funding
The study is fully supported by the National Natural Science Foundation of China (Grant No. 52072412), the Changsha Science and Technology Project (Grant No. KQ1707017), and the innovation driven project of the Central South University (2019CX005).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
ESM 1
(PDF 127 kb)
Rights and permissions
About this article
Cite this article
Li, Y., Liu, Z. & Liu, H. A novel ensemble reinforcement learning gated unit model for daily PM2.5 forecasting. Air Qual Atmos Health 14, 443–453 (2021). https://doi.org/10.1007/s11869-020-00948-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11869-020-00948-x