Fast incremental expectation maximization (FIEM) is a version of the EM framework for large datasets. In this paper, we first recast FIEM and other incremental EM type algorithms in the Stochastic Approximation within EM framework. Then, we provide nonasymptotic bounds for the convergence in expectation as a function of the number of examples n and of the maximal number of iterations \(K_\mathrm {max}\). We propose two strategies for achieving an \(\epsilon \)-approximate stationary point, respectively with \(K_\mathrm {max}= O(n^{2/3}/\epsilon )\) and \(K_\mathrm {max}= O(\sqrt{n}/\epsilon ^{3/2})\), both strategies relying on a random termination rule before \(K_\mathrm {max}\) and on a constant step size in the Stochastic Approximation step. Our bounds provide some improvements on the literature. First, they allow \(K_\mathrm {max}\) to scale as \(\sqrt{n}\) which is better than \(n^{2/3}\) which was the best rate obtained so far; it is at the cost of a larger dependence upon the tolerance \(\epsilon \), thus making this control relevant for small to medium accuracy with respect to the number of examples n. Second, for the \(n^{2/3}\)-rate, the numerical illustrations show that thanks to an optimized choice of the step size and of the bounds in terms of quantities characterizing the optimization problem at hand, our results design a less conservative choice of the step size and provide a better control of the convergence in expectation.

快速增量期望最大化 (FIEM) 是用于大型数据集的 EM 框架的一个版本。在本文中，我们首先在 EM框架内的Stochastic Approximation 中重铸了 FIEM 和其他增量 EM 类型的算法。然后，我们为期望收敛提供非渐近边界，作为示例数n和最大迭代次数\(K_\mathrm {max}\)的函数。我们提出了两种策略来实现\(\epsilon \) -近似驻点，分别为\(K_\mathrm {max}= O(n^{2/3}/\epsilon )\)和\(K_\mathrm {max}= O(\sqrt{n}/\epsilon ^{3/2})\)，这两种策略都依赖于之前的随机终止规则\(K_\mathrm {max}\)和随机近似步骤中的恒定步长。我们的界限对文献提供了一些改进。首先，它们允许\(K_\mathrm {max}\)缩放为\(\sqrt{n}\)这比\(n^{2/3}\)更好，这是迄今为止获得的最佳速率；它的代价是对容差\(\epsilon \) 的依赖性更大，因此使这种控制与示例数量n 的中小精度相关。二、对于\(n^{2/3}\)-rate，数值插图表明，由于步长和边界的优化选择，表征了手头优化问题的数量，我们的结果设计了一个不太保守的步长选择，并提供了更好的控制期望收敛。