First-Order Frameworks for Continuous Newton-like Dynamics Governed by Maximally Monotone Operators

Abstract

In a Hilbert framework, we discuss a continuous Newton-like model that is well-adapted in view to numerical purposes for solving convex minimization and more general monotone inclusion problems. Algorithmic solutions to these problems were recently inspired by implicit temporal discretizations of the (stabilized) continuous version of Nesterov’s accelerated gradient method with an additional Hessian damping term (so as to attenuate the oscillation effects). Unfortunately, due to the presence of the Hessian term, these discrete variants require several gradients or proximal evaluations (per iteration). An alternative methodology can be realized by means of a first-order model that no more involves the Hessian term and that can be extended to the case of an arbitrary maximally monotone operator. Our first-order model originates from the reformulation of a closely related variant to the Nesterov-like equation. Its dynamics are studied (simultaneously) with regard to convex minimization and monotone inclusion problems by considering it when governed by the sum of the gradient of a convex differentiable function and (up to a multiplicative constant) the Yosida approximation of a maximally monotone operator, with an appropriate adjustment of the regularization parameter. It turns out that our model, offers a new framework for discrete variants, while keeping the main asymptotic features of the (stabilized) Nesterov-like equation. Two new algorithms are then suggested relative to the considered optimization problems.

Appendix

1.1 A.1 The Yosida Approximation

The Yosida approximation enjoys numerous nice properties which often facilitating and simplifying calculations. Some of them are recalled below (see [12, 13]):

Proposition A.1

Let $A : {\mathscr{H}} \to 2^{{\mathscr{H}}}$ be a maximal monotone operator and set J_λA := (I + λA)^− 1. Then, we have the following properties :

• (a) J_λA is single-valued, everywhere defined and nonexpansive,

• (b) $\forall v\in {\mathscr{H}}, \forall \lambda >0, A_{\lambda } v \in A(J_{\lambda A}v),$

• (c) $\forall (u,v)\in {\mathscr{H}}^{2}, \forall \lambda >0, ~\langle A_{\lambda } u - A_{\lambda } v, u - v \rangle \geq \lambda \|A_{\lambda } u - A_{\lambda } v\|^{2},$

• (d) $\forall (u,v)\in {\mathscr{H}}^{2}, \forall \lambda >0, ~\| A_{\lambda } u - A_{\lambda } v\| \leq \frac {1}{\lambda }\|u - v\|,$

• (e) $\forall \lambda >0, A^{-1}(\{0\}) = A_{\lambda }^{-1}(\{0\}),$

• (f) ∀λ > 0,∀μ > 0, (A_λ)_μ = A_λ+μ.

Lemma A.1

Let γ,δ > 0 and $x, y \in {\mathscr{H}}$. Then for z ∈ A^− 1({0}), we have

$$ \begin{array}{@{}rcl@{}} && \|\gamma A_{\gamma}x-\delta A_{\delta}y \| \leq 2\|x-y\| + 2\frac{|\gamma - \delta|}{\gamma}\|x-z\|, \end{array} $$

(A.1)

$$ \begin{array}{@{}rcl@{}} && \| A_{\gamma}x- A_{\delta}y \| \le \left ( 3\frac{| \delta-\gamma|}{\delta\gamma} \times \|x-z\| + \frac{2}{\delta}\|x-y\|\right ). \end{array} $$

(A.2)

Proof

The proof of (A.1) can be found in [8]. Let us prove (A.2). To get this we simply have

$\begin {array}{l} A_{\gamma }x- A_{\delta }y= \frac {1}{ \delta } \left (\delta A_{\gamma }x- \delta A_{\delta }y\right ) \\ \hskip 2cm = \frac {1}{ \delta } \left ((\delta -\gamma ) A_{\gamma }x + (\gamma A_{\gamma }x -\delta A_{\delta }y\right ), \end {array}$

hence

$\begin {array}{l} \| A_{\gamma }x- A_{\delta }y \| \le \frac {1}{ \delta } \left (| \delta -\gamma | \times \|A_{\gamma }x \|+ \| \gamma A_{\gamma }x -\delta A_{\delta }y\| \right ). \end {array}$

Consequently, by $\|A_{\gamma }x\|\le \frac {1}{\gamma } \|x-z\|$ and using (A.1), we obtain

$\begin {array}{l} \| A_{\gamma }x- A_{\delta }y \| \le \frac {1}{ \delta } \left (\frac {| \delta -\gamma |}{\gamma } \times \|x-z\| + 2\|x-y\| + 2\frac {|\gamma - \delta |}{\gamma }\|x-z\| \right ) \\ \hskip 2.cm = \frac {1}{ \delta } \left (3\frac {| \delta -\gamma |}{\gamma } \times \|x-z\| + 2\|x-y\| \right ), \end {array}$ that is the desired inequality □

1.2 A.2 A Technical Result

Lemma A.2 (8 Lemma A.5)

Let $\omega ,\eta : [0,+\infty [ \to [0,+\infty [$ be absolutely continuous functions such that $\eta \notin L^{1}(0,+\infty )$ and which satisfy ${\int \limits }_{0}^{+\infty } \omega (t)\eta (t) dt < \infty$, along with $|\dot {\omega }(t)|\leq \eta (t)$ for almost every t > 0. Then $\displaystyle \lim _{t\to +\infty } \omega (t) = 0$.