Random forests have been one of the successful ensemble algorithms in machine learning, and the basic idea is to construct a large number of random trees individually and make predictions based on an average of their predictions. The great successes have attracted much attention on theoretical understandings of random forests, mostly focusing on regression problems. This work takes one step towards the convergence rates of random forests for classification. We present the first finite-sample rate $O(n^{-1/(8d+2)})$ on the convergence of purely random forests for binary classification, which can be improved to be of $O(n^{-1/(3.87d+2)})$ by considering the midpoint splitting mechanism. We introduce another variant of random forests, which follows Breiman's original random forests but with different mechanisms on splitting dimensions and positions. We present the convergence rate $O(n^{-1/(d+2)}{(\ln n)}^{1/(d+2)})$ for the variant of random forests, which reaches the minimax rate, except for a factor ${(\ln n)}^{1/(d+2)}$, of the optimal plug-in classifier under the L-Lipschitz assumption. We achieve the tighter convergence rate $O(\sqrt{\ln n/n})$ under some assumptions over structural data. This work also takes one step towards the convergence rate of random forests for multi-class learning, and presents the same convergence rates of random forests for multi-class learning as that of binary classification, yet with different constants. We finally provide empirical studies to support the theoretical analysis.

随机森林一直是机器学习中成功的集成算法之一，其基本思想是单独构建大量随机树，并根据它们的预测平均值进行预测。巨大的成功引起了对随机森林理论理解的广泛关注，主要集中在回归问题上。这项工作朝着随机森林分类的​​收敛速度迈出了一步。我们提出了第一个有限采样率$○(n^{-1/(8d+2)})$关于二元分类的纯随机森林收敛，可以改进为$○(n^{-1/(3.87d+2)})$通过考虑中点分裂机制。我们介绍了随机森林的另一种变体，它遵循 Breiman 的原始随机森林，但在分割维度和位置方面具有不同的机制。我们提出收敛速度$○(n^{-1/(d+2)}{(\ln n)}^{1/(d+2)})$对于随机森林的变体，它达到极小值率，除了一个因子${(\ln n)}^{1/(d+2)}$, L - Lipschitz 假设下的最优插件分类器。我们实现了更紧密的收敛速度$○(\sqrt{\ln n/n})$在对结构数据的一些假设下。这项工作还朝着多类学习随机森林的收敛速度迈出了一步，并提出了多类学习随机森林的收敛速度与二元分类相同，但常数不同。我们最终提供了实证研究来支持理论分析。