Abstract

Time series modeling has entered an era of unprecedented growth in the size and complexity of data which require new modeling approaches. While many new general purpose machine learning approaches have emerged, they remain poorly understand and irreconcilable with more traditional statistical modeling approaches. We present a general class of exponential smoothed recurrent neural networks (RNNs) which are well suited to modeling nonstationary dynamical systems arising in industrial applications. In particular, we analyze their capacity to characterize the nonlinear partial autocorrelation structure of time series and directly capture dynamic effects such as seasonality and trends. Application of exponentially smoothed RNNs to forecasting electricity load, weather data, and stock prices highlight the efficacy of exponential smoothing of the hidden state for multistep time series forecasting. The results also suggest that popular, but more complicated neural network architectures originally designed for speech processing are likely over-engineered for industrial forecasting and light-weight exponentially smoothed architectures, trained in a fraction of the time, capture the salient features while being superior and more robust than simple RNNs and autoregressive models. Additionally, uncertainty quantification of Bayesian exponential smoothed RNNs is shown to provide improved coverage.

摘要

时间序列建模已经进入了一个数据规模和复杂性空前增长的时代，这需要新的建模方法。尽管出现了许多新的通用机器学习方法，但它们仍然难以理解，并且与更传统的统计建模方法不相容。我们提出了一类通用的指数平滑循环神经网络 (RNN)，它们非常适合对工业应用中出现的非平稳动态系统进行建模。特别是，我们分析了它们表征时间序列的非线性偏自相关结构并直接捕获诸如季节性和趋势等动态效应的能力。指数平滑 RNN 在预测电力负荷、天气数据、和股票价格突出了隐藏状态的指数平滑对多步时间序列预测的功效。结果还表明，最初为语音处理而设计的流行但更复杂的神经网络架构可能过度设计用于工业预测和轻量级指数平滑架构，在很短的时间内进行训练，在捕捉显着特征的同时具有优越性和比简单的 RNN 和自回归模型更健壮。此外，贝叶斯指数平滑 RNN 的不确定性量化被证明可以提供更好的覆盖率。但是最初为语音处理设计的更复杂的神经网络架构可能会过度设计用于工业预测和轻量级指数平滑架构，在很短的时间内进行训练，捕获显着特征，同时比简单的 RNN 和自回归更出色和更健壮楷模。此外，贝叶斯指数平滑 RNN 的不确定性量化被证明可以提供更好的覆盖率。但是最初为语音处理设计的更复杂的神经网络架构可能会过度设计用于工业预测和轻量级指数平滑架构，在很短的时间内进行训练，捕获显着特征，同时比简单的 RNN 和自回归更出色和更健壮楷模。此外，贝叶斯指数平滑 RNN 的不确定性量化被证明可以提供更好的覆盖率。