Abstract
Weather forecasting is a challenging task, which is especially suited for artificial intelligence due to the large amount of data involved. This paper proposed an end-to-end hybrid regression model, called Ensemble of Spatial-Temporal Attention Network and Multi-Layer Perceptron (E-STAN-MLP), to forecast surface temperature, humidity, wind speed, and wind direction at 24 automatic weather stations in Beijing. Combining the data from historical observations with the data from the numerical weather prediction (NWP) system, our proposed model give better results than the NWP system or previously reported algorithms. Our E-STAN-MLP model consists of two parts. One is to use the spatial-temporal attention based recurrent neural network to model the time series of meteorological elements. The other is a simple but efficient multi-layer perceptron architecture forecasts the regression value while ignoring time dependence. Results at each time stamp are integrated together using a step-wise fusion strategy. Moreover, we use a joint loss step integrating both the regression loss function and the classification loss function to simultaneously forecast the wind speed and direction. Experiments demonstrate that our proposed E-STAN-MLP model achieves state-of-the-art results in weather forecasting.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv:1409.0473 (2014)
Chen, N., Qian, Z., Nabney, I.T., Meng, X.: Short-term wind power forecasting using gaussian processes. In: Twenty-third international joint conference on artificial intelligence (2013)
Chen, T., Guestrin, C.: Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp 785–794 (2016)
Grover, A., Kapoor, A., Horvitz, E.: A deep hybrid model for weather forecasting. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 379–386 (2015)
Guo, S., Lin, Y., Feng, N., Song, C., Wan, H.: Attention based Spatial-Temporal graph convolutional networks for traffic flow forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp 922–929 (2019)
Hernández, E., Sanchez-Anguix, V., Julian, V., Palanca, J., Duque, N.: Rainfall prediction: A deep learning approach. In: International Conference on Hybrid Artificial Intelligence Systems, pp 151–162 (2016)
Lai, G., Chang, W.-C., Yang, Y., Liu, H.: Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp 95–104 (2018)
Liang, Y., Ke, S., Zhang, J., Yi, X., Zheng, Y.: GeoMAN: Multi-level Attention Networks for Geo-sensory Time Series Prediction. In: IJCAI, pp 3428–3434 (2018)
Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988 (2017)
Liu, H., Tian, H.-Q., Li, Y.-F.: Comparison of two new ARIMA-ANN and ARIMA-kalman hybrid methods for wind speed prediction. Appl. Energy 98, 415–424 (2012)
Lorenz, E.N.: Deterministic nonperiodic flow, journal of the atmospheric sciences vol. 20, No. In. XX (1963)
Luo, P., Ren, J., Peng, Z., Zhang, R., Li, J.: Differentiable learning-to-normalize via switchable normalization. arXiv:1806.10779 (2018)
Lynch, P.: Weather prediction by numerical process weather prediction by numerical process (2006)
Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings icml, p 3 (2013)
Marchuk, G.: Numerical methods in weather prediction. Numer. Meth. Weather Predict. 259–273 (1974)
Percival, D.B., Walden, A.T.: Spectral analysis for physical applications. Cambridge University Press, Cambridge (1993)
Shih, S.-Y., Sun, F.-K., Lee, H.-Y.: Temporal pattern attention for multivariate time series forecasting. Mach Learn 108 (2019)
Song, D., Chen, H., Jiang, G., Qin, Y.: Dual Stage Attention Based Recurrent Neural Network for Time Series Prediction. In: Google Patents (2018)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112 (2014)
Taylor, S., Letham, B.: Forecasting at scale. The American Statistician 72 (2017)
Tolstykh, M., Frolov, A.: Some current problems in numerical weather prediction. Izve. Atmos. Ocean. Phys. 41, 285–295 (2005)
Voyant, C., Muselli, M., Paoli, C., Nivet, M. -L.: Numerical Weather Prediction (NWP) and hybrid ARMA/ANN model to predict global radiation. Energy 39 (2012)
Wang, B., Lu, J., Yan, Z., Luo, H., Li, T., Zheng, Y., Zhang, G.: Deep Uncertainty Quantification: A Machine Learning Approach for Weather Forecasting. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 2087–2095. ACM, New York (2019)
Xie, Y., Fan, S., Chen, M., Shi, J., Zhong, J., Zhang, X.: An assessment of satellite radiance data assimilation in RMAPS. Remote Sens 11(1), 54 (2019)
Xu, W., Peng, H., Zeng, X., Zhou, F., Tian, X., Peng, X.: A hybrid modelling method for time series forecasting based on a linear regression model and deep learning. Appl Intell 1–14 (2019)
Acknowledgements
This work was partially supported by the National Key Research and Development Program of China (grant numbers 2018YFC1506801, 2018YFF0300102, 2018YFC1407506); National Natural Science Foundation of China (grant number 11621101); the Ministry of Science and Technology of China (grant number IUMKY201904) and Fundamental Research Funds for the Central Universities (grant number 2019FZA5002,
Zhejiang University NGICS Platform). We would also like to thank Drs. Julian Evans and Xinhua Zhu for their valuable comments and discussions. Thanks also be given to Drs. Min Chen, Mingxuan Chen, and Shiguang Miao from IUM for their great supports to our research project.
Author information
Authors and Affiliations
Corresponding authors
Additional information
Communicated by: Soon-Il An
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Li, Y., Lang, J., Ji, L. et al. Weather Forecasting Using Ensemble of Spatial-Temporal Attention Network and Multi-Layer Perceptron. Asia-Pacific J Atmos Sci 57, 533–546 (2021). https://doi.org/10.1007/s13143-020-00212-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13143-020-00212-3