SIAM Journal on Imaging Sciences, Volume 13, Issue 4, Page 2029-2062, January 2020.
Following the seminal work of Nesterov, accelerated optimization methods have been used to powerfully boost the performance of first-order, gradient based parameter estimation in scenarios where second-order optimization strategies are either inapplicable or impractical. Not only does accelerated gradient descent converge considerably faster than traditional gradient descent, but it also performs a more robust local search of the parameter space by initially overshooting and then oscillating back as it settles into a final configuration, thereby selecting only local minimizers with a basis of attraction large enough to contain the initial overshoot. This behavior has made accelerated and stochastic gradient search methods particularly popular within the machine learning community. In their recent PNAS 2016 paper, A Variational Perspective on Accelerated Methods in Optimization, Wibisono, Wilson, and Jordan demonstrate how a broad class of accelerated schemes can be cast in a variational framework formulated around the Bregman divergence, leading to continuum limit ODEs. We show how their formulation may be further extended to infinite dimensional manifolds (starting here with the geometric space of curves and surfaces) by substituting the Bregman divergence with inner products on the tangent space and explicitly introducing a distributed mass model which evolves in conjunction with the object of interest during the optimization process. The coevolving mass model, which is introduced purely for the sake of endowing the optimization with helpful dynamics, also links the resulting class of accelerated PDE based optimization schemes to fluid dynamical formulations of optimal mass transport.

中文翻译：

---

主动轮廓情况下 PDE 框架公式的加速优化


SIAM 影像科学杂志，第 13 卷，第 4 期，第 2029-2062 页，2020 年 1 月。

继 Nesterov 的开创性工作之后，加速优化方法已被用于在二阶优化策略不适用或不切实际的情况下，有力地提高一阶、基于梯度的参数估计的性能。加速梯度下降不仅收敛速度比传统梯度下降快得多，而且它还通过最初的超调，然后在稳定到最终配置时振荡回来，对参数空间执行更鲁棒的局部搜索，从而仅选择具有基的局部最小化器吸引力足够大以包含初始超调。这种行为使得加速和随机梯度搜索方法在机器学习社区中特别流行。 Wibisono、Wilson 和 Jordan 在最近的 PNAS 2016 论文《优化中加速方法的变分视角》中展示了如何在围绕 Bregman 散度制定的变分框架中构建广泛的加速方案，从而得出连续极限 ODE。我们展示了如何通过用切空间上的内积代替布雷格曼散度并明确引入一个分布式质量模型，将其公式进一步扩展到无限维流形（从曲线和曲面的几何空间开始）。优化过程中感兴趣的对象。引入协同演化质量模型纯粹是为了赋予优化有用的动力学，还将基于加速偏微分方程的优化方案的结果类别与最佳质量传递的流体动力学公式联系起来。