In recent years, the online schema of the conventional Random Forests(RFs) have attracted much attention because of its ability to handle sequential data or data whose distribution changes during the prediction process. However, most research on online RFs focuses on structural modification during the training stage, overlooking critical aspects of the sequential dataset, such as autocorrelation. In this paper, we demonstrate how to improve the predictive accuracy of the regression model by exploiting data correlation. Instead of modifying the structure of the off-line trained RFs, we endow RFs with memory during regression prediction through an online weight learning approach, which is called Online Weight Learning Random Forest Regression(OWL-RFR). Specifically, the weights of leaves are updated based on a novel adaptive stochastic gradient descent method, in which the adaptive learning rate considers the current and historical prediction bias ratios, compared with the static learning rate. Thus, leaf-level weight stores the learned information from the past data points for future correlated prediction. Compared with tree-level weight which only has immediate memory for current prediction, the leaf-level weight can provide long-term memory. Numerical experiments with OWL-RFR show remarkable improvements in predictive accuracy across several common machine learning datasets, compared to traditional RFs and other online approaches. Moreover, our results verify that the weight approach using the long-term memory of leaf-level weight is more effective than immediate dependency on tree-level weight. We show the improved effectiveness of the proposed adaptive learning rate in comparison to the static rate for most datasets, we also show the convergence and stability of our method.

近年来，常规随机森林（RF）的在线模式已吸引了很多关注，因为它具有处理顺序数据或分布在预测过程中发生变化的数据的能力。但是，大多数在线RF的研究都集中在训练阶段的结构修改上，忽略了顺序数据集的关键方面，例如自相关。在本文中，我们演示了如何通过利用数据相关性来提高回归模型的预测准确性。我们没有修改离线训练的RF的结构，而是通过在线权重学习方法（称为在线权重学习随机森林回归（OWL-RFR））在回归预测过程中为RF赋予了记忆。特别，根据一种新颖的自适应随机梯度下降方法更新叶子的权重，该方法中的自适应学习率与静态学习率相比考虑了当前和历史预测偏差率。因此，叶级权重存储了从过去的数据点学来的信息，以用于将来的相关预测。与仅对当前预测具有即时记忆的树级权重相比，叶级权重可以提供长期记忆。与传统的RF和其他在线方法相比，使用OWL-RFR进行的数字实验表明，在多个常见的机器学习数据集中，预测准确性有了显着提高。此外，我们的结果验证了使用叶级权重的长期存储的权重方法比直接依赖于树级权重更有效。