We theoretically analyze the information geometric optimization (IGO), which is a unified framework of stochastic search algorithms for black-box optimization. The IGO framework has two parameters: 1) the learning rate and 2) the sample size, and they influence the behavior of the algorithm. We investigate the strategy parameters of the IGO with the family of isotropic Gaussian distributions on a general convex quadratic function. Compared to the previous theoretical works, where an infinite sample size is assumed and the deterministic algorithm dynamics is studied, we investigate the expected improvement of the algorithm with a finite sample size. The analysis finds that the relative decrease rates of the distance from the distribution mean to the landscape optimum and the distribution standard deviation must be the same, which we observe in practice, while the analysis based on an infinite sample size failed to obtain. We derive these rates explicitly as a function of the eigenvalues of the Hessian of the objective function and the strategy parameters. We also derive the stable value of the ratio of the square distance to the optimum over the distribution variance, as well as the conditions that the stable value exists. These theoretical values coincide with our numerical simulations.

中文翻译：

---

凸二次函数上具有各向同性高斯分布的信息几何优化的有限样本分析

我们从理论上分析了信息几何优化（IGO），它是用于黑盒优化的随机搜索算法的统一框架。IGO 框架有两个参数：1) 学习率和 2) 样本大小，它们影响算法的行为。我们使用一般凸二次函数上的各向同性高斯分布族研究 IGO 的策略参数。与先前假设无限样本量并研究确定性算法动力学的理论工作相比，我们研究了具有有限样本量的算法的预期改进。分析发现，从分布均值到景观最优值的距离的相对减少率和分布标准差必须相同，我们在实践中观察到，而基于无限样本量的分析未能获得。我们将这些比率明确地推导出为目标函数的 Hessian 特征值和策略参数的函数。我们还推导出平方距离与最佳分布方差之比的稳定值，以及稳定值存在的条件。这些理论值与我们的数值模拟一致。