Skip to main content
Log in

A novel ensemble reinforcement learning gated unit model for daily PM2.5 forecasting

  • Published:
Air Quality, Atmosphere & Health Aims and scope Submit manuscript

Abstract

PM2.5 forecasting is an important scientific way to control environmental pollution and keep people healthy. To achieve high-performance PM2.5 forecasting, a new ensemble reinforcement learning gated unit model is presented in this study. The complete framework of this model mainly includes the following steps: In step I, the WPD method is applied to decompose PM2.5 data into 8 sub-series with different frequency types. In step II, the SAE-GRU method is presented to finish the establishment of sub-series forecasting model. Among them, SAE is used to obtain low-latitude features of PM2.5 data, and GRU is applied to finish PM2.5 sub-series forecasting. In step III, Q-learning is used to combine the every PM2.5 sub-series to get the final model prediction results. By comparing and analyzing the final results of all case study, it can be summarized that (1) Q-learning-based ensemble model integrates the subseries with different frequency types perfectly, and results prove that it is better than heuristic algorithm, and (2) the proposed ensemble reinforcement learning gated unit model can get prediction results beyond seventeen alternative models which include three most state-of-the-art models in all cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Adelia R, Suyanto S, Wisesty UN (2019) Indonesian abstractive text summarization using bidirectional gated recurrent unit. Procedia Comput Sci 157:581–588. https://doi.org/10.1016/j.procs.2019.09.017

    Article  Google Scholar 

  • Adem K, Kiliçarslan S, Cömert O (2019) Classification and diagnosis of cervical cancer with stacked autoencoder and softmax classification. Expert Syst Appl 115:557–564. https://doi.org/10.1016/j.eswa.2018.08.050

    Article  Google Scholar 

  • Bai Y, Zeng B, Li C, Zhang J (2019) An ensemble long short-term memory neural network for hourly PM2.5 concentration forecasting. Chemosphere 222:286–294

    Article  CAS  Google Scholar 

  • Becerra-Rico J, Aceves-Fernández MA, Esquivel-Escalante K, Pedraza-Ortega JC (2020) Airborne particle pollution predictive model using Gated Recurrent Unit (GRU) deep neural networks. Earth Science Informatics:1–14

  • Biancofiore F, Busilacchio M, Verdecchia M, Tomassetti B, Aruffo E, Bianco S, Di Tommaso S, Colangeli C, Rosatelli G, Di Carlo P (2017) Recursive neural network model for analysis and forecast of PM10 and PM2. 5. Atmospheric Pollution Research 8(4):652–659

    Article  Google Scholar 

  • Błaszczyk E, Rogula-Kozłowska W, Klejnowski K, Kubiesa P, Fulara I, Mielżyńska-Švach D (2017) Indoor air quality in urban and rural kindergartens: short-term studies in Silesia, Poland. Air Qual Atmos Health 10(10):1207–1220

    Article  Google Scholar 

  • Cheng Y, Zhang H, Liu Z, Chen L, Wang P (2019) Hybrid algorithm for short-term forecasting of PM2. 5 in China. Atmos Environ 200:264–279

    Article  CAS  Google Scholar 

  • Ding Y, Ma L, Ma J, Suo M, Tao L, Cheng Y, Lu C (2019) Intelligent fault diagnosis for rotating machinery using deep Q-network based health state classification: a deep reinforcement learning approach. Adv Eng Inform 42:100977. https://doi.org/10.1016/j.aei.2019.100977

    Article  Google Scholar 

  • Doreswamy KSH, Km Y, Gad I (2020) Forecasting air pollution particulate matter (PM2.5) using machine learning regression models. Procedia Comput Sci 171:2057–2066. https://doi.org/10.1016/j.procs.2020.04.221

    Article  Google Scholar 

  • Hähnel P, Mareček J, Monteil J, O'Donncha F (2020) Using deep learning to extend the range of air pollution monitoring and forecasting. J Comput Phys 408:109278. https://doi.org/10.1016/j.jcp.2020.109278

    Article  CAS  Google Scholar 

  • Haidong S, Junsheng C, Hongkai J, Yu Y, Zhantao W (2020) Enhanced deep gated recurrent unit and complex wavelet packet energy moment entropy for early fault prognosis of bearing. Knowl-Based Syst 188:105022. https://doi.org/10.1016/j.knosys.2019.105022

    Article  Google Scholar 

  • Jiang P, Li C, Li R, Yang H (2019) An innovative hybrid air pollution early-warning system based on pollutants forecasting and Extenics evaluation. Knowl-Based Syst 164:174–192. https://doi.org/10.1016/j.knosys.2018.10.036

    Article  Google Scholar 

  • Kristjanpoller W, Minutolo MC (2015) Gold price volatility: a forecasting approach using the artificial neural network–GARCH model. Expert Syst Appl 42(20):7245–7251. https://doi.org/10.1016/j.eswa.2015.04.058

    Article  Google Scholar 

  • Kroon M, Whiteson S (2009) Automatic feature selection for model-based reinforcement learning in factored MDPs. In: International Conference on Machine Learning and Applications, ICMLA 2009, Miami Beach, Florida, USA, December 13-15, 2009

  • Liu H, Duan Z, Chen C (2019a) A hybrid framework for forecasting PM2. 5 concentrations using multi-step deterministic and probabilistic strategy. Air Qual Atmos Health 12(7):785–795

    Article  CAS  Google Scholar 

  • Liu H, Duan Z, Wu H, Li Y, Dong S (2019b) Wind speed forecasting models based on data decomposition, feature selection and group method of data handling network. Measurement 148:106971

    Article  Google Scholar 

  • Liu H, Jin K, Duan Z (2019c) Air PM2. 5 concentration multi-step forecasting using a new hybrid modeling method: comparing cases for four cities in China. Atmos Pollut Res 10(5):1588–1600

    Article  CAS  Google Scholar 

  • Liu H, Xu Y, Chen C (2019d) Improved pollution forecasting hybrid algorithms based on the ensemble method. Appl Math Model 73:473–486. https://doi.org/10.1016/j.apm.2019.04.032

    Article  Google Scholar 

  • Liu H, Duan Z, Chen C (2020a) A hybrid multi-resolution multi-objective ensemble model and its application for forecasting of daily PM2.5 concentrations. Inf Sci 516:266–292. https://doi.org/10.1016/j.ins.2019.12.054

    Article  Google Scholar 

  • Liu H, Long Z, Duan Z, Shi H (2020b) A new model using multiple feature clustering and neural networks for forecasting hourly PM2. 5 concentrations, and its applications in China. Engineering

  • Liu H, Yu C, Wu H, Duan Z, Yan G (2020c) A new hybrid ensemble deep reinforcement learning model for wind speed short term forecasting. Energy 202:117794. https://doi.org/10.1016/j.energy.2020.117794

    Article  Google Scholar 

  • Liu H, Yu C, Yu C, Chen C, Wu H (2020d) A novel axle temperature forecasting method based on decomposition, reinforcement learning optimization and neural network. Adv Eng Inform 44:101089. https://doi.org/10.1016/j.aei.2020.101089

    Article  Google Scholar 

  • Mirza AH, Kerpicci M, Kozat SS (2020) Efficient online learning with improved LSTM neural networks. Digit Signal Process 102:102742. https://doi.org/10.1016/j.dsp.2020.102742

    Article  Google Scholar 

  • Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533

    Article  CAS  Google Scholar 

  • Momennejad I, Russek EM, Cheong JH, Botvinick MM, Daw ND, Gershman SJ (2017) The successor representation in human reinforcement learning. Nat Hum Behav 1(9):680–692

    Article  CAS  Google Scholar 

  • Ocak H, Loparo KA, Discenzo FM (2007) Online tracking of bearing wear using wavelet packet decomposition and probabilistic modeling: a method for bearing prognostics. J Sound Vib 302(4–5):951–961

    Article  Google Scholar 

  • Peduzzi E, Pisoni E, Clappier A, Thunis P (2018) Multi-level policies for air quality: implications of national and sub-national emission reductions on population exposure. Air Qual Atmos Health 11(9):1121–1135

    Article  CAS  Google Scholar 

  • Sharma N, Taneja S, Sagar V, Bhatt A (2018) Forecasting air pollution load in Delhi using data analysis tools. Procedia Comput Sci 132:1077–1085. https://doi.org/10.1016/j.procs.2018.05.023

    Article  Google Scholar 

  • Solaiman T, Coulibaly P, Kanaroglou P (2008) Ground-level ozone forecasting using data-driven methods. Air Qual Atmos Health 1(4):179–193

    Article  CAS  Google Scholar 

  • Song J, Wang J, Lu H (2018) A novel combined model based on advanced optimization algorithm for short-term wind speed forecasting. Appl Energy 215:643–658

    Article  Google Scholar 

  • Xiong R, Cao J, Yu Q (2018) Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle. Appl Energy 211:538–548. https://doi.org/10.1016/j.apenergy.2017.11.072

    Article  Google Scholar 

  • Xu Y, Liu H (2020) Spatial ensemble prediction of hourly PM2. 5 concentrations around Beijing railway station in China. Air Qual Atmos Health:1–11

  • Xu F, Yang F, Fan X, Huang Z, Tsui KL (2020a) Extracting degradation trends for roller bearings by using a moving-average stacked auto-encoder and a novel exponential function. Measurement 152:107371. https://doi.org/10.1016/j.measurement.2019.107371

    Article  Google Scholar 

  • Xu Y, Liu H, Duan Z (2020b) A novel hybrid model for multi-step daily AQI forecasting driven by air pollution big data. Air Qual Atmos Health 13(2):197–207

    Article  CAS  Google Scholar 

  • Yerramilli A, Dodla VBR, Challa VS, Myles L, Pendergrass WR, Vogel CA, Dasari HP, Tuluri F, Baham JM, Hughes RL (2012) An integrated WRF/HYSPLIT modeling approach for the assessment of PM 2.5 source regions over the Mississippi Gulf Coast region. Air Qual Atmos Health 5(4):401–412

    Article  CAS  Google Scholar 

  • Yin P-Y, Chao C-H (2018) Automatic selection of fittest energy demand predictors based on cyber swarm optimization and reinforcement learning. Appl Soft Comput 71:152–164. https://doi.org/10.1016/j.asoc.2018.06.042

    Article  Google Scholar 

  • Zabalza J, Ren J, Zheng J, Zhao H, Qing C, Yang Z, Du P, Marshall S (2016) Novel segmented stacked autoencoder for effective dimensionality reduction and feature extraction in hyperspectral imaging. Neurocomputing 185:1–10

    Article  Google Scholar 

Download references

Funding

The study is fully supported by the National Natural Science Foundation of China (Grant No. 52072412), the Changsha Science and Technology Project (Grant No. KQ1707017), and the innovation driven project of the Central South University (2019CX005).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hui Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(PDF 127 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Liu, Z. & Liu, H. A novel ensemble reinforcement learning gated unit model for daily PM2.5 forecasting. Air Qual Atmos Health 14, 443–453 (2021). https://doi.org/10.1007/s11869-020-00948-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11869-020-00948-x

Keywords

Navigation