当前位置: X-MOL 学术Math. Program. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
First-order optimization algorithms via inertial systems with Hessian driven damping
Mathematical Programming ( IF 2.2 ) Pub Date : 2020-11-16 , DOI: 10.1007/s10107-020-01591-1
Hedy Attouch , Zaki Chbani , Jalal Fadili , Hassan Riahi

In a Hilbert space setting, for convex optimization, we analyze the convergence rate of a class of first-order algorithms involving inertial features. They can be interpreted as discrete time versions of inertial dynamics involving both viscous and Hessian-driven dampings. The geometrical damping driven by the Hessian intervenes in the dynamics in the form $\nabla^2 f (x(t)) \dot{x} (t)$. By treating this term as the time derivative of $ \nabla f (x (t)) $, this gives, in discretized form, first-order algorithms in time and space. In addition to the convergence properties attached to Nesterov-type accelerated gradient methods, the algorithms thus obtained are new and show a rapid convergence towards zero of the gradients. On the basis of a regularization technique using the Moreau envelope, we extend these methods to non-smooth convex functions with extended real values. The introduction of time scale factors makes it possible to further accelerate these algorithms. We also report numerical results on structured problems to support our theoretical findings.

中文翻译:

通过具有 Hessian 驱动阻尼的惯性系统的一阶优化算法

在希尔伯特空间设置中,对于凸优化,我们分析了一类涉及惯性特征的一阶算法的收敛速度。它们可以解释为惯性动力学的离散时间版本,包括粘性阻尼和 Hessian 驱动阻尼。由 Hessian 驱动的几何阻尼以 $\nabla^2 f (x(t)) \dot{x} (t)$ 的形式干预动力学。通过将此项视为 $\nabla f (x (t)) $ 的时间导数,这以离散形式给出了时间和空间中的一阶算法。除了附加到 Nesterov 型加速梯度方法的收敛特性之外,由此获得的算法是新的并且显示向梯度的零快速收敛。基于使用 Moreau 包络的正则化技术,我们将这些方法扩展到具有扩展实数值的非光滑凸函数。时间比例因子的引入使得进一步加速这些算法成为可能。我们还报告了结构化问题的数值结果以支持我们的理论发现。
更新日期:2020-11-16
down
wechat
bug