Gradient-descent-based algorithms and their stochastic versions have widespread applications in machine learning and statistical inference. In this work, we carry out an analytic study of the performance of the algorithm most commonly considered in physics, the Langevin algorithm, in the context of noisy high-dimensional inference. We employ the Langevin algorithm to sample the posterior probability measure for the spiked mixed matrix-tensor model. The typical behavior of this algorithm is described by a system of integrodifferential equations that we call the Langevin state evolution, whose solution is compared with the one of the state evolution of approximate message passing (AMP). Our results show that, remarkably, the algorithmic threshold of the Langevin algorithm is suboptimal with respect to the one given by AMP. This phenomenon is due to the residual glassiness present in that region of parameters. We also present a simple heuristic expression of the transition line, which appears to be in agreement with the numerical results.

中文翻译：

---

嘈杂的高维推理中Langevin算法的奇迹和陷阱

基于梯度下降的算法及其随机版本在机器学习和统计推断中具有广泛的应用。在这项工作中，我们在嘈杂的高维推理环境下，对物理学中最常考虑的算法Langevin算法的性能进行了分析研究。我们采用Langevin算法对加标混合矩阵张量模型的后验概率度量进行采样。该算法的典型行为由一个积分微分方程系统（我们称为Langevin状态演化）描述，该系统的解与近似消息传递（AMP）的状态演化之一进行比较。我们的结果表明，与AMP给出的阈值相比，Langevin算法的算法阈值次优。这种现象是由于在该参数区域中存在残留的玻璃光泽。我们还提出了过渡线的一种简单的启发式表达，这似乎与数值结果一致。