当前位置: X-MOL 学术Math. Program. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
First-order inertial algorithms involving dry friction damping
Mathematical Programming ( IF 2.7 ) Pub Date : 2021-01-13 , DOI: 10.1007/s10107-020-01613-y
Samir Adly , Hedy Attouch

In a Hilbert space $$ {\mathcal H}$$ H , we introduce a new class of first-order algorithms which naturally occur as discrete temporal versions of an inertial differential inclusion jointly involving viscous friction and dry friction. The function $$f:{\mathcal H}\rightarrow {\mathbb {R}}$$ f : H → R to be minimized is supposed to be differentiable (not necessarily convex), and enters the algorithm via its gradient. The dry friction damping function $$\phi :{\mathcal H}\rightarrow {\mathbb {R}}_+$$ ϕ : H → R + is convex with a sharp minimum at the origin, (typically $$\phi (x) = r \Vert x\Vert $$ ϕ ( x ) = r ‖ x ‖ with $$r >0$$ r > 0 ). It enters the algorithm via its proximal mapping, which acts as a soft threshold operator on the velocities. As a result, we obtain a new class of splitting algorithms involving separately the proximal and gradient steps. The sequence of iterates has a finite length, and therefore strongly converges towards an approximate critical point $$x_{\infty }$$ x ∞ of f (typically $$\Vert \nabla f(x_{\infty })\Vert \le r$$ ‖ ∇ f ( x ∞ ) ‖ ≤ r ). Under a geometric property satisfied by the limit point $$x_{\infty }$$ x ∞ , we obtain geometric and finite rates of convergence. The convergence results tolerate the presence of errors, under the sole assumption of their asymptotic convergence towards zero. By replacing the function f by its Moreau envelope, we extend the results to the case of nonsmooth convex functions. In this case, the algorithm involves the proximal operators of f and $$\phi $$ ϕ separately. Several variants of this algorithm are considered, including the case of the Nesterov accelerated gradient method. We then consider the extension in the case of additive composite optimization, thus leading to new splitting methods. Numerical experiments are given for Lasso-type problems. The performance profiles, as a comparison tool, demonstrate the efficiency of the Nesterov accelerated method with asymptotic vanishing damping combined with dry friction.

中文翻译:

涉及干摩擦阻尼的一阶惯性算法

在希尔伯特空间 $$ {\mathcal H}$$ H 中,我们引入了一类新的一阶算法,这些算法自然地作为惯性微分包含的离散时间版本出现,联合涉及粘性摩擦和干摩擦。函数 $$f:{\mathcal H}\rightarrow {\mathbb {R}}$$ f : H → R 要最小化应该是可微的(不一定是凸的),并通过其梯度进入算法。干摩擦阻尼函数 $$\phi :{\mathcal H}\rightarrow {\mathbb {R}}_+$$ ϕ : H → R + 是凸的,在原点处有一个尖锐的最小值,(通常为 $$\phi (x) = r \Vert x\Vert $$ ϕ ( x ) = r ‖ x ‖ 其中 $$r >0$$ r > 0 )。它通过其近端映射进入算法,该映射充当速度的软阈值算子。因此,我们获得了一类新的分裂算法,分别涉及近端和梯度步骤。迭代序列具有有限长度,因此强烈收敛于 f 的近似临界点 $$x_{\infty }$$ x ∞ (通常为 $$\Vert \nabla f(x_{\infty })\Vert \ le r$$ ‖ ∇ f ( x ∞ ) ‖ ≤ r )。在极限点 $$x_{\infty }$$ x ∞ 满足的几何性质下,我们获得了几何收敛速度和有限收敛速度。收敛结果容忍误差的存在,在它们渐近收敛到零的唯一假设下。通过用它的 Moreau 包络替换函数 f,我们将结果扩展到非光滑凸函数的情况。在这种情况下,算法分别涉及 f 和 $$\phi $$ ϕ 的近端算子。考虑了该算法的几种变体,包括 Nesterov 加速梯度法的情况。然后我们考虑在加性复合优化的情况下的扩展,从而导致新的分裂方法。数值实验给出了套索型问题。性能曲线作为比较工具,证明了具有渐近消失阻尼和干摩擦的 Nesterov 加速方法的效率。
更新日期:2021-01-13
down
wechat
bug