Optimal sampled-data controls with running inequality state constraints: Pontryagin maximum principle and bouncing trajectory phenomenon

Abstract

In the present paper we derive a Pontryagin maximum principle for general nonlinear optimal sampled-data control problems in the presence of running inequality state constraints. We obtain, in particular, a nonpositive averaged Hamiltonian gradient condition associated with an adjoint vector being a function of bounded variation. As a well known challenge, theoretical and numerical difficulties may arise due to the possible pathological behavior of the adjoint vector (jumps and singular part lying on parts of the optimal trajectory in contact with the boundary of the restricted state space). However, in our case with sampled-data controls, we prove that, under certain general hypotheses, the optimal trajectory activates the running inequality state constraints at most at the sampling times. Due to this so-called bouncing trajectory phenomenon, the adjoint vector experiences jumps at most at the sampling times (and thus in a finite number and at precise instants) and its singular part vanishes. Taking advantage of these informations, we are able to implement an indirect numerical method which we use to solve three simple examples.

Appendices

Preliminaries for the proof of Theorem 3.1

This appendix is devoted to some required preliminaries for the proof of Theorem 3.1 found in Appendix B. In Sect. A.1 we give some recalls on renorming Banach spaces and on the regularity of distance functions. Sect. A.2 is concerned with the sensitivity analysis of the state equation in Problem ($\mathrm {OSCP}$). Then, in Sect. A.3, we give some recalls on Stieltjes integrations and on Fubini formulas. Finally Sect. A.4 is devoted to Duhamel formulas for Cauchy-Stieltjes Problems (FCSP) and (BCSP).

1.1 About renorming Banach spaces and regularity of distance functions

Let $(Y, \Vert \cdot \Vert )$ be a normed space. We recall that the dual space of $(Y,\Vert \cdot \Vert )$, which we denote by $Y^*:={\mathcal {L}}((Y,\Vert \cdot \Vert ),{\mathbb {R}})$, is the space of linear continuous forms on $(Y,\Vert \cdot \Vert )$. We recall that $Y^*$ can be endowed with the dual norm $\Vert \cdot \Vert ^{*}$ defined by

$$\begin{aligned} \begin{array}{lrcl}\Vert \cdot \Vert ^* :&{}Y^* &{}\longrightarrow &{}{\mathbb {R}}_+\\ {} &{}y^*&{} \longmapsto &{}\Vert y^* \Vert ^* := \sup \limits _{\begin{array}{c} y\in Y\\ \Vert y \Vert \le 1 \end{array}} | \langle y^* ,y\rangle _{Y^* \times Y} | . \end{array} \end{aligned}$$

In this situation we denote by $(Y^*,\Vert \cdot \Vert ^*):=\mathrm {dual}(Y,\Vert \cdot \Vert )$. We recall the following proposition on renorming separable Banach spaces.

Proposition A.1

Let $(Y,\Vert \cdot \Vert )$ be a separable Banach space and let $(Y^*, \Vert \cdot \Vert ^*)=\mathrm {dual}(Y,\Vert \cdot \Vert )$. Then there exists a norm $\mathrm {N}$ on Y equivalent to $\Vert \cdot \Vert$ such that:

• (i) $\mathrm {N}^*$ is equivalent to $\Vert \cdot \Vert ^*$;

• (ii) $\mathrm {N}^*$ is strictly convex;

where $(Y^*,\mathrm {N}^*)=\mathrm {dual}(Y,\mathrm {N})$.

Proof

We refer to [56, Theorem 2.18, p. 42] or to [15, Proposition 4, p. 16] for a complete proof. $\square$

Let $F : Y \rightarrow {\mathbb {R}}$ be a convex function. Recall that the subdifferential of F at a point $y \in Y$ is defined to be the set

$$\begin{aligned} \partial F(y):= \{y^* \in Y^* \mid \langle y^*, y'-y \rangle _{Y^* \times Y} \le F(y') -F(y) \text { for all } y'\in Y\}. \end{aligned}$$

We recall that a function $F : Y \rightarrow {\mathbb {R}}$ is said to be strictly Hadamard-differentiable at a point $y \in Y$ with the strict Hadamard derivative $DF ( y ) \in Y^*$ if

$$\begin{aligned} \lim \limits _{\begin{array}{c} y' \rightarrow y \\ t \searrow 0 \end{array}} \left[ \; \sup _{y'' \in \mathrm {K}} \left| \frac{F(y'+t y'')-F(y')}{t}-\langle DF (y) ,y'' \rangle _{Y^* \times Y}\right| \right] =0, \end{aligned}$$

for every compact set $\mathrm {K} \subset Y$. We refer to [62, pp. 312–313] for more details. Finally we denote by $\mathrm {d_S} : Y \rightarrow {\mathbb {R}}$ the distance function to a nonempty subset $\mathrm {S} \subset Y$ defined by $\mathrm {d_S}(y):=\inf _{y' \in \mathrm {S}}\Vert y-y' \Vert$ for all $y \in Y$, and by $\mathrm {d^2_S} : Y \rightarrow {\mathbb {R}}$ the squared distance function defined by $\mathrm {d^2_S}(y) := \mathrm {d_S} (y)^2$ for all $y \in Y$. We recall the following proposition on the regularity of distance functions.

Proposition A.2

Let $(Y, \Vert \cdot \Vert )$ be a normed space. Let $\mathrm {S}\subset Y$ be a nonempty closed convex subset and let us assume that $\Vert \cdot \Vert ^*$ is strictly convex, where $(Y^*,\Vert \cdot \Vert ^*):=\mathrm {dual}(Y,\Vert \cdot \Vert )$. Then it holds that:

• (i) $\mathrm {d_S}$ is convex and 1-Lipschitz continuous;

• (ii) $\mathrm {d_S}$ is strictly Hadamard-differentiable on $Y {\setminus }\mathrm {S}$ with $\Vert D\mathrm {d_S}(y)\Vert ^*=1$ and $\partial \mathrm {d_S}(y)=\{D\mathrm {d_S}(y)\}$ for all $y\in Y{\setminus } \mathrm {S}$;

• (iii) $\mathrm {d^2_S}$ is strictly Hadamard-differentiable on $Y {\setminus }\mathrm {S}$ with $D\mathrm {d^2_S} (y)=2\mathrm {d_S}(y)D \mathrm {d_S}(y)$ for all $y\in Y{\setminus }\mathrm {S}$;

• (iv) $\mathrm {d^2_S}$ is Fréchet-differentiable on $\mathrm {S}$ with $D\mathrm {d^2_S}(y)=0_{Y^*}$ for all $y\in \mathrm {S}$.

Proof

The proof of (i) is a standard result. We refer to [62, Theorem 3.54, p. 313] and [15, Appendix B.2] for the proof of (ii). The proofs of (iii) and (iv) are straightforward. $\square$

1.2 About sensitivity analysis of the state equation in Problem ($\mathrm {OSCP}$)

For all $u\in \mathrm {L}^\infty _m$ we consider the Cauchy problem ($\mathrm {CP}_u$) given by

Before proceeding to the sensitivity analysis of the Cauchy problem ($\mathrm {CP}_u$) with respect to the control u, we first recall some definitions and results from the classical Cauchy–Lipschitz (or Picard-Lindelöf) theory (see e.g., [28]).

Definition A.1

Let $u\in \mathrm {L}^\infty _m$. A (local) solution to the Cauchy problem ($\mathrm {CP}_u$) is a couple (x, I) such that:

• (i) I is an interval such that $\{ 0 \} \varsubsetneq I \subset [0,T]$;

• (ii) $x \in \mathrm {AC}([0,T'],{\mathbb {R}}^n)$, with ${\dot{x}}(t) = f(x(t),u(t),t)$ for a.e. $t \in [0,T']$, for all $T' \in I$;

• (iii) $x(0)=x_0$.

Let $(x_1,I_1)$ and $(x_2,I_2)$ be two (local) solutions to the Cauchy problem ($\mathrm {CP}_u$). We say that $(x_2,I_2)$ is an extension (resp. strict extension) to $(x_1,I_1)$ if $I_1\subset I_2$ (resp. $I_1 \varsubsetneq I_2$) and $x_2(t) = x_1(t)$ for all $t \in I_1$. A maximal solution to the Cauchy problem ($\mathrm {CP}_u$) is a (local) solution that does not admit any strict extension. Finally a global solution to the Cauchy problem ($\mathrm {CP}_u$) is a solution (x, I) such that $I=[0,T]$.

Proposition A.3

For all $u\in \mathrm {L}^\infty _m$, the Cauchy problem ($\mathrm {CP}_u$) admits a unique maximal solution, denoted by $(x(\cdot ,u),I(u))$, which is an extension to any other local solution.

We now introduce the notion of controls admissible for globality.

Definition A.1

A control $u\in \mathrm {L}^\infty _m$ is said to be admissible for globality if the corresponding maximal solution $(x(\cdot ,u),I(u))$ is global, that is, if $I(u) = [0,T]$. In what follows we denote by $\mathcal {AG}\subset \mathrm {L}^\infty _m$ the set of all controls admissible for globality.

Remark A.1

Using the standard combination of the Gronwall lemma with the blow-up theorem for nonglobal solutions in ordinary differential equations theory, we can establish the following sufficient condition. Given a control $u\in \mathrm {L}^\infty _m$, if there exist a nonnegative coercive mapping $\Theta : {\mathbb {R}}^n \rightarrow {\mathbb {R}}_+$ of class $\mathrm {C}^1$ with two nonnegative constants $c_1$, $c_2 \ge 0$ such that $\langle f(x,u(t),t) , \nabla \Theta ( x ) \rangle _{{\mathbb {R}}^n} \le c_1 \Theta (x) + c_2$ for all $x \in {\mathbb {R}}^n$ and for a.e. $t \in [0,T]$, then $u \in \mathcal {AG}$. This sufficient condition covers, not only some typical situations for which $\mathcal {AG} = \mathrm {L}^\infty _m$ (such as global Lipschitz dynamics, or more generally dynamics with a sublinear growth, taking $\Theta (x) := \Vert x \Vert _{{\mathbb {R}}^n}^2$ for all $x \in {\mathbb {R}}^n$), but also some dynamics with polynomial growth for which $\mathcal {AG} \subsetneq \mathrm {L}^\infty _m$. As an illustration, take the scalar case $n=m=1$ and the dynamics $f(x,u,t):=x-ux^3$ for all $(x,u,t) \in {\mathbb {R}}\times {\mathbb {R}}\times [0,T]$. In that example, if a scalar control $u \in \mathrm {L}^\infty _m$ takes only nonnegative values on [0, T], by considering $\Theta (x) := x^2$ for all $x \in {\mathbb {R}}$, we prove that $u \in \mathcal {AG}$.

In the following lemma we state a continuous dependence result for the trajectory $x(\cdot ,u)$ with respect to the control u. In particular we prove that $\mathcal {AG}$ is open.

Lemma A.1

For all $u\in \mathcal {AG}$, there exists $\varepsilon _u>0$ such that $\overline{\mathrm {B}}_{\mathrm {L}^\infty _m}(u,\varepsilon _u)\subset \mathcal {AG}$, where $\overline{\mathrm {B}}_{\mathrm {L}^\infty _m}(u,\varepsilon _u)$ stands for the standard closed ball in $\mathrm {L}^\infty _m$ centered at u and of radius $\varepsilon _u$. Moreover the map

$$\begin{aligned} u'\in (\overline{\mathrm {B}}_{\mathrm {L}^\infty _m}(u,\varepsilon _u),\Vert \cdot \Vert _{\mathrm {L}^\infty _m})\longmapsto x(\cdot ,u')\in (\mathrm {C}_n,\Vert \cdot \Vert _{\infty }), \end{aligned}$$

is Lipschitz continuous.

Proof

This proof is standard and essentially based on the classical Gronwall lemma. We refer to [17, Lemmas 1 and 3, pp. 3795–3797], [19, Lemmas 4.3 and 4.5, pp. 73–74] (in the general framework of time scale calculus) or to [15, Propositions 1 and 2, pp. 4–5] (in a more classical framework, closer to the present considerations) for similar statements with detailed proofs. $\square$

Remark A.2

Let $u\in \mathcal {AG}$ and $\varepsilon _u>0$ as given in Lemma A.1. Let $u' \in \overline{\mathrm {B}}_{\mathrm {L}^\infty _m}(u,\varepsilon _u)$ and $(u_k)_{k\in {\mathbb {N}}}$ be a sequence in $\overline{\mathrm {B}}_{\mathrm {L}^\infty _m}(u,\varepsilon _u)$ converging to $u'$ in $\mathrm {L}^\infty _m$. From Lemma A.1, we deduce that the sequence $(x(\cdot ,u_k))_{k\in {\mathbb {N}}}$ uniformly converges to $x(\cdot ,u')$ over [0, T].

In the next proposition we state a differentiability result for the trajectory $x(\cdot ,u)$ with respect to a convex $\mathrm {L}^\infty$-perturbation of the control u.

Proposition A.4

Let $u\in \mathcal {AG}$ and let $z \in \mathrm {L}^\infty _m$. We consider the convex $\mathrm {L}^\infty$-perturbation of u given by

$$\begin{aligned} u_z(\cdot ,\rho ) := u + \rho (z-u), \end{aligned}$$

for all $\rho \in [0,1]$. Then:

• (i) there exists $0<\rho _0\le 1$ such that $u_z(\cdot ,\rho )\in \mathcal {AG}$ for all $\rho \in [0,\rho _0]$;

• (ii) the map

  $$\begin{aligned} \rho \in ([0,\rho _0],\vert \cdot \vert )\longmapsto x(\cdot ,u_z(\cdot ,\rho ))\in (\mathrm {C}_n,\Vert \cdot \Vert _\infty ), \end{aligned}$$

  is differentiable at $\rho =0$ and its derivative is equal to the variation vector $w_z(\cdot ,u) \in \mathrm {AC}_n$ being the unique solution (that is global) to the linearized Cauchy problem given by

  $$\begin{aligned} {\left\{ \begin{array}{ll} {\dot{w}}(t) = \partial _1 f(x(t,u),u(t),t)\times w(t)+\partial _2 f(x(t,u),u(t),t)\times (z(t)-u(t)) \\ \qquad \text { for a.e. }t \in [0,T], \\ w(0)=0_{{\mathbb {R}}^n}. \end{array}\right. } \end{aligned}$$

Proof

This proof is standard and essentially based on the classical Gronwall lemma. We refer to [17, Lemma 4 and Proposition 1, pp. 3797–3798] for a similar statement with detailed proof. $\square$

We conclude this section by a technical lemma on the convergence of variation vectors which is required in the proof of our main result.

Lemma A.2

Let $u\in \mathcal {AG}$ and $\varepsilon _u>0$ as in Lemma A.1. Let $z \in \mathrm {L}^\infty _m$. Let $u' \in \overline{\mathrm {B}}_{\mathrm {L}^\infty _m}(u,\varepsilon _u)$ and $(u_k)_{k\in {\mathbb {N}}}$ be a sequence in $\overline{\mathrm {B}}_{\mathrm {L}^\infty _m}(u,\varepsilon _u)$ converging to $u'$ in $\mathrm {L}^\infty _m$. Then the sequence $(w_z(\cdot ,u_k))_{k\in {\mathbb {N}}}$ uniformly converges to $w_z(\cdot ,u')$ over [0, T].

Proof

This proof is standard and essentially based on the classical Gronwall lemma. We refer to [17, Lemmas 4.8 and 4.9, pp. 77–78] for a similar statement with detailed proof. $\square$

1.3 About Stieltjes integrations and Fubini formulas

In this section our aim is to recall some notions on Stieltjes integrations and to recall some Fubini formulas. We refer to standard references and books such as [5, 22, 23, 41, 75] for more details. We also refer to [15, Appendix C] and references therein. In the sequel we denote by $\mathrm {C}^+_n:=\mathrm {C}([0,T],({\mathbb {R}}_{+})^n)$ where ${\mathbb {R}}_{+}:=[0,+\infty )$. We denote by $\mathrm {C}^*_n$ as the dual space of $\mathrm {C}_n$ (see Sect. A.1 for some details on dual spaces). We first recall the following Riesz representation theorem (see [57, Theorem 14.5, pp. 245–246] or [15, Proposition 7, p. 19]).

Proposition A.5

$\text {(Riesz representation theorem).}$ Let $\psi ^* \in \mathrm {C}^*_1$. Then there exists a unique $\eta \in \mathrm {NBV}_1$ such that:

$$\begin{aligned} \langle \psi ^* , \psi \rangle _{\mathrm {C}^*_1\times \mathrm {C}_1}=\displaystyle \int _0^T \psi (\tau ) \, d\eta (\tau ), \end{aligned}$$

for all $\psi \in \mathrm {C}_1$. Moreover it holds that:

• (i) $\langle \psi ^* , \psi \rangle _{\mathrm {C}^*_1\times \mathrm {C}_1}\ge 0$ for all $\psi \in \mathrm {C}^{+}_1$ if and only if $\eta$ is monotonically increasing on [0, T];

• (ii) $\psi ^* = 0_{\mathrm {C}^*_1}$ if and only if $\eta =0_{\mathrm {NBV}_1}$.

Recall that if $\eta \in \mathrm {NBV}_1$ is monotonically increasing on [0, T], then $\eta$ induces a finite nonnegative Borel measure $d\eta$ on [0, T] by defining $d\eta (\{ 0 \}) := \eta (0^+)$ and $d\eta ((a,b]):=\eta (b)-\eta (a)$ for all semiopen intervals $(a,b] \subset [0,T]$ and by using the Carathéodory extension theorem. Furthermore, for all $\psi \in \mathrm {C}_1$, the Riemann–Stieltjes integral $\int _a^b \psi (\tau ) d\eta (\tau )$ coincides with the Lebesgue–Stieltjes integral $\int _{(a,b]} \psi (\tau ) d\eta (\tau )$ for all $0 \le a \le b \le T$. We refer to [41, p. 83] and [75, p. 288] for more details. Consequently the Fubini formula

$$\begin{aligned} \int _0^T \int _{0}^{\tau } \Psi (\tau , s) \,ds \, d\eta (\tau ) = \int _0^T \int _{s}^T \Psi (\tau ,s) \,d\eta (\tau ) \,ds, \end{aligned}$$

(4)

holds for all $\Psi \in \mathrm {L}^\infty ([0,T]^2,{\mathbb {R}})$ such that $\Psi$ is continuous in its first variable.

We now introduce some notations for Riemann–Stieltjes integrals with respect to vectorial functions of bounded variation. We denote by

$$\begin{aligned} \int _0^T \langle \psi (\tau ), d \eta (\tau )\rangle := \sum _{j=1}^{q} \int _0^T \psi _j(\tau )\, d\eta _j(\tau ) \in {\mathbb {R}}, \end{aligned}$$

for all $\psi =(\psi _j)_{j=1,\ldots ,q}\in \mathrm {C}_{q}$ and all $\eta = (\eta _j)_{j=1,\ldots ,q}\in \mathrm {BV}_{q}$. Moreover we denote by

$$\begin{aligned} \int _0^T M(\tau ) \times d\eta (\tau ):= \left( \sum _{j=1}^{q} \int _0^T m_{r j }(\tau )\,d\eta _j(\tau ) \right) _{r=1,\ldots ,n} \in {\mathbb {R}}^{n}, \end{aligned}$$

and

$$\begin{aligned} \int _0^T \langle \psi (\tau ), M(\tau ) \times d\eta (\tau ) \rangle := \int _0^T \langle M(\tau )^\top \times \psi (\tau ), d\eta (\tau )\rangle \in {\mathbb {R}}, \end{aligned}$$

for all $\psi =(\psi _r)_{r=1,\ldots ,n}\in \mathrm {C}_{n}$, all $\eta = (\eta _j)_{j=1,\ldots ,q}\in \mathrm {BV}_{q}$ and all continuous matrices $M=(m_{r j})_{r j} : [0,T] \rightarrow {\mathbb {R}}^{n \times q}$. In particular one can easily prove that, if $\psi \in {\mathbb {R}}^{n}$ (i.e. $\psi \in \mathrm {C}_{n}$ constant), then

$$\begin{aligned} \int _0^T \langle \psi , M(\tau )\times d\eta (\tau )\rangle = \left\langle \psi , \int _0^T M(\tau ) \times d\eta (\tau ) \right\rangle _{{\mathbb {R}}^{n}}, \end{aligned}$$

for all $\eta = (\eta _j)_{j=1,\ldots ,q}\in \mathrm {BV}_{q}$ and all continuous matrices $M=(m_{r j})_{r j} : [0,T] \rightarrow {\mathbb {R}}^{n \times q}$.

Finally, from the Fubini formula (4) and the above notations, one can easily deduce that the Fubini formula

$$\begin{aligned} \int _0^T \left\langle \int _0^\tau \Psi (\tau ,s) \,ds, d\eta (\tau )\right\rangle = \int _0^T \int _s^T \langle \Psi (\tau ,s), d\eta (\tau )\rangle \,ds, \end{aligned}$$

holds for all $\Psi \in \mathrm {L}^\infty ([0,T]^2,{\mathbb {R}}^{q})$ being continuous in its first variable and for all $\eta = (\eta _j)_{j=1,\ldots ,q}\in \mathrm {NBV}_{q}$ such that $\eta _j$ is monotonically increasing on [0, T] for each $j=1,\ldots ,q$.

1.4 About problems (FCSP) and (BCSP) and Duhamel formulas

Let us consider the framework and the notations introduced in Sect. 2.2. Our aim in this section is to provide Duhamel formulas for the solutions to Problems (FCSP) and (BCSP). To this aim, we recall that the state-transition matrix $\Phi (\cdot ,\cdot ) : [0,T]^2 \rightarrow {\mathbb {R}}^{n \times n}$ associated with $A\in \mathrm {L}^\infty ([0,T],{\mathbb {R}}^{n \times n})$ is defined as follows. For all $s\in [0,T]$, $\Phi (\cdot ,s)$ is the unique solution (that is global) to the linear forward/backward Cauchy problem given by

$$\begin{aligned} {\left\{ \begin{array}{ll} {\dot{\Phi }}(t) = A(t) \times \Phi (t) \text { for a.e. } t\in [0,T],\\ \Phi (s)= \mathrm {Id}_n. \end{array}\right. } \end{aligned}$$

The equalities

$$\begin{aligned} \Phi (t,s) = \mathrm {Id}_n + \int _s^t A(\tau ) \times \Phi (\tau ,s)\, d\tau = \mathrm {Id}_n + \int _s^t \Phi (t,\tau ) \times A(\tau )\, d\tau , \end{aligned}$$

both hold for all $(t,s)\in [0,T]^2$. From these two equalities and the Fubini formulas from Sect. A.3, one can easily derive the following proposition. We also refer to [15, Appendix D] for some details.

Proposition A.6

[Duhamel formulas] The solutions to (FCSP) and (BCSP) are given by

$$\begin{aligned} x(t)=\Phi (t,0)\times x_0 + \int _0^t \Phi (t,\tau ) \times B(\tau ) \, d\tau + \sum _{j=1}^q \int _0^t \Phi (t,\tau )\times C_j(\tau ) \,d\eta _j(\tau ), \end{aligned}$$

and

$$\begin{aligned} p(t)=\Phi (T,t)^\top \times p_T + \int _t^T \Phi (\tau ,t)^\top \times B(\tau ) \, d\tau + \sum _{j=1}^q\int _t^T \Phi (\tau ,t)^\top \times C_j(\tau ) \, d\eta _j(\tau ), \end{aligned}$$

for all $t\in [0,T]$, where $\Phi (\cdot ,\cdot )$ stands for the state-transition matrix associated with A.

Proof of Theorem 3.1

This appendix is dedicated to the detailed proof of Theorem 3.1. Section B.1 deals with the case $L = 0$ (the case $L\ne 0$ is treated in Sect. B.2 with a simple change of variable). In Sect. B.1.1 the Ekeland variational principle is applied on an appropriate penalized functional in order to derive a crucial inequality (see Inequality (9)). In Sect. B.1.2 we conclude the proof of Theorem 3.1 by introducing the adjoint vector p.

We first remark that the running inequality state constraints in Problem ($\mathrm {OSCP}$) can be written as ${\mathfrak {h}}(x)\in \mathrm {S}$ where:

• ${\mathfrak {h}}: \mathrm {C}_n\rightarrow \mathrm {C}_q$ is defined as ${\mathfrak {h}}(x) := h(x,\cdot )$ for all $x \in \mathrm {C}_n$. Note that ${\mathfrak {h}}$ is of class $\mathrm {C}^1$ with $D{\mathfrak {h}}(x)(x')=\partial _1 h (x,\cdot ) \times x'$ for all x, $x' \in \mathrm {C}_n$;

• $\mathrm {S}:=\mathrm {C}([0,T],({\mathbb {R}}_-)^q)$ where ${\mathbb {R}}_- := (-\infty ,0]$. We emphasize that $\mathrm {S} \subset \mathrm {C}_q$ is a nonempty closed convex cone of $\mathrm {C}_q$ with a nonempty interior.

Recall that $(\mathrm {C}_q,\Vert \cdot \Vert _\infty )$ is a separable Banach space. Applying Proposition A.1, we endow $\mathrm {C}_q$ with an equivalent norm $\Vert \cdot \Vert _{\mathrm {C}_q}$ such that the associated dual norm $\Vert \cdot \Vert _{\mathrm {C}^*_q}$ is strictly convex. We denote by $\mathrm {d_S} : \mathrm {C}_q \rightarrow {\mathbb {R}}$ the 1-Lipschitz continuous distance function to $\mathrm {S}$ (see Sect. A.1). Then, from Proposition A.2, we know that $\mathrm {d_S}$ and $\mathrm {d^2_S}$ are strictly Hadamard-differentiable on $\mathrm {C}_q {\setminus } \mathrm {S}$ with $D\mathrm {d^2_S}(x)=2\mathrm {d_S}(x)D\mathrm {d_S}(x)$ and $\Vert D\mathrm {d_S}(x) \Vert _{\mathrm {C}_q^*} = 1$ for all $x\in \mathrm {C}_q {\setminus } \mathrm {S}$, and that $\mathrm {d^2_S}$ is Fréchet-differentiable on $\mathrm {S}$ with $D\mathrm {d^2_S}(x)=0_{\mathrm {C}_q^*}$ for all $x\in \mathrm {S}$.

1.1 The case $L=0$

In the whole section we will assume that $L=0$ in Problem ($\mathrm {OSCP}$) (see Sect. B.2 for the case $L \ne 0$). Let $(x,u)\in \mathrm {AC}_n\times \mathrm {PC}^{\mathbb {T}}_m$ be a solution to Problem ($\mathrm {OSCP}$). Following the notation introduced in Sect. A.2, it holds that $u \in \mathcal {AG}$ and that $x=x(\cdot ,u)$. In what follows we will also consider the positive real number $\varepsilon _{u} > 0$ given in Lemma A.1.

1.1.1 Application of the Ekeland variational principle

Let us recall a simplified version (but sufficient for our purposes) of the Ekeland varational principle (see [38]).

Proposition B.1

(Ekeland variational principle) Let $(\mathrm {E},\mathrm {d_E})$ be a complete metric set. Let ${\mathcal {J}}: \mathrm {E} \rightarrow {\mathbb {R}}^+$ be a continuous nonnegative map. Let $\varepsilon >0$ and $e \in \mathrm {E}$ such that ${\mathcal {J}}(e) = \varepsilon$. Then there exists $e_\varepsilon \in \mathrm {E}$ such that $\mathrm {d_E}(e_\varepsilon ,e)\le \sqrt{\varepsilon }$, and $-\sqrt{\varepsilon } \; \mathrm {d_E}(e',e_\varepsilon )\le {\mathcal {J}}(e')-{\mathcal {J}}(e_\varepsilon )$ for every $e' \in \mathrm {E}$.

We introduce the set

$$\begin{aligned} \mathrm {E}_u :=\{u' \in \overline{\mathrm {B}}_{\mathrm {L}^\infty _m}(u,\varepsilon _u) \mid u' \in \mathrm {PC}^{\mathbb {T}}_m \text { and } u'(t) \in \mathrm {U}\text { for all } t\in [0,T]\}. \end{aligned}$$

From the closedness assumption on $\mathrm {U}$, one can easily prove that $(\mathrm {E}_u,\Vert \cdot \Vert _{\mathrm {L}^\infty _m})$ is a complete metric set. Let us choose a sequence $(\varepsilon _k)_{k\in {\mathbb {N}}}$ such that $0<\sqrt{\varepsilon _k}<\varepsilon _u$ for all $k\in {\mathbb {N}}$ and satisfying $\lim _{k\rightarrow \infty }\varepsilon _k=0$. We introduce the penalized functional

$$\begin{aligned} \begin{array}{lrcl}{\mathcal {J}}_k :&{}\mathrm {E}_u &{}\longrightarrow &{}{\mathbb {R}}_+\\ {} &{}u'&{} \longmapsto &{}{\mathcal {J}}_k(u'):=\sqrt{ \Bigg ( \Big ( g(x(T,u'))-g(x(T))+\varepsilon _k \Big )^+\Bigg )^2+\mathrm {d^2_S}\Big ( {\mathfrak {h}}(x(\cdot ,u')) \Big )}, \end{array} \end{aligned}$$

for all $k\in {\mathbb {N}}$. From Lemma A.1, note that ${\mathcal {J}}_k$ is correctly defined for all $k\in {\mathbb {N}}$. Also, from Lemma A.1 and from the continuities of g, ${\mathfrak {h}}$ and $\mathrm {d}^2_\mathrm {S}$ (see Proposition A.2), it follows that ${\mathcal {J}}_k$ is continuous as well for all $k\in {\mathbb {N}}$. Note that ${\mathcal {J}}_k$ is nonnegative and, since the constraint ${\mathfrak {h}}( x) \in \mathrm {S}$ is satisfied, it holds that ${\mathcal {J}}_k(u)=\varepsilon _k$ for all $k\in {\mathbb {N}}$. Therefore, from the Ekeland variational principle (see Proposition B.1), we conclude that there exists a sequence $(u_k)_{k\in {\mathbb {N}}} \subset \mathrm {E}_u$ such that

$$\begin{aligned} \Vert u_k-u \Vert _{\mathrm {L}^\infty _m} \le \sqrt{\varepsilon _k}, \end{aligned}$$

(5)

and

$$\begin{aligned} -\sqrt{\varepsilon _k} \; \Vert u'-u_k \Vert _{\mathrm {L}^\infty _m}\le {\mathcal {J}}_k(u')-{\mathcal {J}}_k(u_k), \end{aligned}$$

(6)

for all $u'\in \mathrm {E}_u$ and all $k\in {\mathbb {N}}$. In particular, from Inequality (5), note that the sequence $(u_k)_{k \in {\mathbb {N}}}$ converges to u in $\mathrm {L}^\infty _m$. From optimality of the couple (x, u), note that ${\mathcal {J}}_k(u')>0$ for all $u'\in \mathrm {E}_u$ and all $k\in {\mathbb {N}}$. We thus define correctly the couple $(\lambda _k,\psi ^*_k) \in {\mathbb {R}} \times \mathrm {C}^*_q$ as

$$\begin{aligned} \lambda _k := \frac{1}{{\mathcal {J}}_k(u_k)} \Big ( g(x(T,u_k))-g(x(T))+\varepsilon _k \Big )^+ \ge 0 , \end{aligned}$$

and

$$\begin{aligned} \psi ^*_k:=\left\{ \begin{array}{lcl} \dfrac{1}{{\mathcal {J}}_k(u_k)}\mathrm {d_S}\Big ( {\mathfrak {h}}(x(\cdot ,u_k))\Big )D\mathrm {d_S}\Big ({\mathfrak {h}}(x(\cdot ,u_k))\Big ) &{} \text {if} &{} {\mathfrak {h}}(x(\cdot ,u_k))\notin \mathrm {S}, \\ 0_{\mathrm {C}^*_q} &{} \text {if} &{} {\mathfrak {h}}(x(\cdot ,u_k)) \in \mathrm {S}, \end{array} \right. \end{aligned}$$

for all $k \in {\mathbb {N}}$. From Proposition A.2 it holds that $|\lambda _k|^2+\Vert \psi ^*_k \Vert ^2_{\mathrm {C}^*_q}=1$ for all $k\in {\mathbb {N}}$. As a consequence, we can extract subsequences (which we do not relabel) such that $(\lambda _k)_{k\in {\mathbb {N}}}$ converges to some $\lambda \ge 0$ and $(\psi ^*_k)_{k\in {\mathbb {N}}}$ weakly$^*$ converges to some $\psi ^* \in \mathrm {C}^*_q$. In particular it holds that $|\lambda |^2+\Vert \psi ^* \Vert ^2_{\mathrm {C}^*_q} \le 1$. At this step note that we cannot ensure that the couple $(\lambda ,\psi ^*)$ is not trivial. The nontriviality is guaranteed by the next proposition.

Proposition B.2

The couple $(\lambda ,\psi ^*)\in {\mathbb {R}}\times \mathrm {C}^*_q$ is nontrivial and it holds that

$$\begin{aligned} \langle \psi ^*, \psi - {\mathfrak {h}}(x) \rangle _{\mathrm {C}^*_q \times \mathrm {C}_q} \le 0, \end{aligned}$$

(7)

for all $\psi \in \mathrm {S}$.

Proof

Let $k\in {\mathbb {N}}$ be fixed. From Proposition A.2, if ${\mathfrak {h}}(x(\cdot ,u_k))\notin \mathrm {S}$, then $D\mathrm {d_S}({\mathfrak {h}}(x(\cdot ,u_k)) ) \in \partial \mathrm {d_S} ({\mathfrak {h}}(x(\cdot ,u_k)) )$. Hence, if ${\mathfrak {h}}(x(\cdot ,u_k))\notin \mathrm {S}$, it holds that

$$\begin{aligned} \left\langle D \mathrm {d_S} \Big ( {\mathfrak {h}}(x(\cdot ,u_k)) \Big ), \psi - {\mathfrak {h}}(x(\cdot ,u_k)) \right\rangle _{\mathrm {C}^*_q\times \mathrm {C}_q} \le \mathrm {d_S}(\psi )-\mathrm {d_S} \Big ( {\mathfrak {h}}(x(\cdot ,u_k) \Big ) \le 0, \end{aligned}$$

for all $\psi \in \mathrm {S}$. As a consequence, in both cases ${\mathfrak {h}}(x(\cdot ,u_k))\in \mathrm {S}$ and ${\mathfrak {h}}(x(\cdot ,u_k))\notin \mathrm {S}$, it holds that

$$\begin{aligned} \big \langle \psi ^*_k, \psi -{\mathfrak {h}}(x(\cdot ,u_k))\big \rangle _{\mathrm {C}^*_q\times \mathrm {C}_q} \le 0, \end{aligned}$$

(8)

for all $\psi \in \mathrm {S}$. Using Lemma A.1 and taking the limit as k tends to $+\infty$, we get Inequality (7). Now let us prove that the couple $(\lambda ,\psi ^*)\in {\mathbb {R}}\times \mathrm {C}^*_q$ is nontrivial. Since $\mathrm {S}$ has a nonempty interior, there exists $\xi \in \mathrm {S}$ and $\delta >0$ such that $\xi +\delta \psi \in \mathrm {S}$ for all $\psi \in \overline{\mathrm {B}}_{\mathrm {C}_q}(0_{\mathrm {C}_q},1)$. Hence we obtain from Inequality (8) that

$$\begin{aligned} \delta \langle \psi ^*_k, \psi \rangle _{\mathrm {C}^*_q \times \mathrm {C}_q} \le \langle \psi ^*_k, {\mathfrak {h}}(x(\cdot ,u_k))- \xi \rangle _{\mathrm {C}^*_q\times \mathrm {C}_q}, \end{aligned}$$

for all $\psi \in \overline{\mathrm {B}}_{\mathrm {C}_q} (0_{\mathrm {C}_q},1)$ and all $k \in {\mathbb {N}}$. We deduce that

$$\begin{aligned} \delta \Vert \psi ^*_k \Vert _{\mathrm {C}^*_q}=\delta \sqrt{1-\vert \lambda _k \vert ^2} \le \langle \psi ^*_k, {\mathfrak {h}}(x(\cdot ,u_k))- \xi \rangle _{\mathrm {C}^*_q\times \mathrm {C}_q}, \end{aligned}$$

for all $k \in {\mathbb {N}}$. Using Lemma A.1 and taking the limit as k tends to $+\infty$, we obtain that

$$\begin{aligned} \delta \sqrt{1-\vert \lambda \vert ^2} \le \langle \psi ^*, {\mathfrak {h}}(x) - \xi \rangle _{\mathrm {C}^*_q\times \mathrm {C}_q}. \end{aligned}$$

Since $\delta >0$, the last inequality implies that the couple $(\lambda ,\psi ^*)$ is nontrivial which completes the proof. $\square$

Finally, in the next result, we use Inequality (6) with convex $\mathrm {L}^\infty$-perturbations of the control $u_k$ in order to establish a crucial inequality.

Proposition B.3

The inequality

$$\begin{aligned} \lambda \Big \langle \nabla g(x(T)),w_z(T,u) \Big \rangle _{{\mathbb {R}}^n} + \Big \langle \psi ^*,\partial _1 h(x,\cdot )\times w_z(\cdot ,u) \Big \rangle _{\mathrm {C}^*_q \times \mathrm {C}_q} \ge 0, \end{aligned}$$

(9)

holds for all $z \in \mathrm {PC}^{\mathbb {T}}_m$ with values in $\mathrm {U}$, where $w_z(\cdot ,u)$ is the variation vector defined in Proposition A.4.

Proof

Let $z \in \mathrm {PC}^{\mathbb {T}}_m$ with values in $\mathrm {U}$. We fix $k \in {\mathbb {N}}$. Since $\mathrm {U}$ is convex, it is clear that the convex $\mathrm {L}^\infty$-pertubation of the control $u_k$ associated with z, defined by $u_{k,z}(t,\rho ):= u_k(t) +\rho (z(t)-u_k(t))$ for all $t\in [0,T]$ and all $0\le \rho \le 1$, belongs to $\mathrm {PC}^{\mathbb {T}}_m$ and takes values in $\mathrm {U}$. Furthermore it holds that $\Vert u_{k,z}(\cdot ,\rho )-u \Vert _{\mathrm {L}^\infty _m} \le \rho \Vert z-u_k \Vert _{\mathrm {L}^\infty _m}+\Vert u_k-u \Vert _{\mathrm {L}^\infty _m} \le \rho \Vert z-u_k \Vert _{\mathrm {L}^\infty _m} + \sqrt{\varepsilon _k}$. Since $\sqrt{\varepsilon _k} < \varepsilon _u$, we deduce that $u_{k,z}(\cdot ,\rho ) \in \mathrm {E}_u$ for small enough $\rho >0$. From Inequality (6) we get that

$$\begin{aligned} -\sqrt{\varepsilon _k} \; \Vert z-u_k \Vert _{\mathrm {L}^\infty _m} \le \frac{1}{{\mathcal {J}}_k(u_{k,z}(\cdot ,\rho ))+{\mathcal {J}}_k(u_k)}\times \frac{{\mathcal {J}}_k(u_{k,z}(\cdot ,\rho ))^2-{\mathcal {J}}_k(u_k)^2}{\rho }, \end{aligned}$$

for small enough $\rho >0$. From Proposition A.4 and from strict Hadamard-differentiability of $\mathrm {d}_\mathrm {S}^2$ over $\mathrm {C}_q {\setminus } \mathrm {S}$ and Fréchet-differentiability of $\mathrm {d}_\mathrm {S}^2$ over $\mathrm {S}$ (see Proposition A.2), taking the limit as $\rho$ tends to 0, we get that

$$\begin{aligned} \begin{aligned}&-\sqrt{\varepsilon _k} \; \Vert z-u_k \Vert _{\mathrm {L}^\infty _m} \le \frac{1}{2 {\mathcal {J}}_k(u_k)} \\&\quad \left[ 2 \Big ( g(x(T,u_k))-g(x(T))+\varepsilon _k \Big )^+ \Big \langle \nabla g(x(T,u_k)),w_z(T,u_k) \Big \rangle _{{\mathbb {R}}^n} \right. \\&\quad \left. + \Big \langle 2 \mathrm {d_S}({\mathfrak {h}}(x(\cdot ,u_k)))D\mathrm {d_S}({\mathfrak {h}}(x(\cdot ,u_k))),\partial _1 h(x(\cdot ,u_k),\cdot )\times w_z(\cdot ,u_k) \Big \rangle _{\mathrm {C}^*_q\times \mathrm {C}_q} \right] , \end{aligned} \end{aligned}$$

with the convention that the second term on the right-hand side is zero if ${\mathfrak {h}}(x(\cdot ,u_k))\in \mathrm {S}$. Using the definition of $\lambda _k$ and $\psi ^*_k$, we deduce that

$$\begin{aligned}&-\sqrt{\varepsilon _k} \; \Vert z-u_k \Vert _{\mathrm {L}^\infty _m} \le \lambda _k \Big \langle \nabla g(x(T,u_k)),w_z(T,u_k) \Big \rangle _{{\mathbb {R}}^n}\\&\quad + \Big \langle \psi ^*_k,\partial _1 h(x(\cdot ,u_k),\cdot )\times w_z(\cdot ,u_k) \Big \rangle _{\mathrm {C}^*_q \times \mathrm {C}_q} . \end{aligned}$$

We take the limit of this inequality as k tends to $+\infty$. From the smoothness of g and h and from Lemmas A.1 and A.2, Inequality (9) is proved. $\square$

1.1.2 Introduction of the adjoint vector

We can now conclude the proof of Theorem 3.1 (in the case $L=0$) by introducing the adjoint vector p. We refer to Sects. 2.2, A.3 and A.4 for notations and background concerning Stieltjes integrations and linear Cauchy–Stieltjes problems.

Introduction of the nontrivial couple $(p^0,\eta )$ and complementary slackness condition. We introduce $p^0 := -\lambda \le 0$ and we write $\psi ^*=(\psi ^*_j)_{j=1,\ldots ,q}$ where $\psi ^*_j\in \mathrm {C}^*_1$ for every $j=1,\ldots ,q$. From the Riesz representation theorem (see Proposition A.5), there exists a unique $\eta _j \in \mathrm {NBV}_1$ such that

$$\begin{aligned} \langle \psi ^*_j, \psi \rangle _{\mathrm {C}^*_1 \times \mathrm {C}_1} = \displaystyle \int _0^T \psi (\tau )\,d\eta _j(\tau ), \end{aligned}$$

for all $\psi \in \mathrm {C}_1$ and all $j=1,\ldots ,q$. Furthermore $\psi ^*_j=0_{\mathrm {C}^*_1}$ if and only if $\eta _j=0_{\mathrm {NBV}_1}$. Thus it follows from Proposition B.2 that the couple $(p^0,\eta )$ is not trivial, where $\eta := (\eta _j)_{j=1,\ldots ,q} \in \mathrm {NBV}_q$. Moreover, from Inequality (7) (and the fact that $\mathrm {S}$ is a cone containing ${\mathfrak {h}}(x)$), one can easily deduce that $\langle \psi ^*_j, {\mathfrak {h}}_j(x)\rangle _{\mathrm {C}^*_1 \times \mathrm {C}_1}=0$, that is,

$$\begin{aligned} \int _0^T h_j(x(\tau ),\tau )\,d\eta _j(\tau )=0, \end{aligned}$$

for all $j=1,\ldots ,q$. Finally one can similarly deduce from Inequality (7) that $\langle \psi ^*_j, \psi \rangle _{\mathrm {C}^*_1 \times \mathrm {C}_1} \ge 0$ for all $\psi \in \mathrm {C}^+_1$ and all $j=1,\ldots ,q$. From Proposition A.5, it follows that $\eta _j$ is monotonically increasing on [0, T] for all $j=1,\ldots ,q$.

Adjoint equation. We define the adjoint vector $p\in \mathrm {BV}_n$ as the unique solution to the backward linear Cauchy–Stieltjes problem given by

$$\begin{aligned} {\left\{ \begin{array}{ll} -dp= \Big ( \partial _1 f(x,u,\cdot )^\top \times p \Big ) \,dt - \sum _{j=1}^{q} \partial _1 h_j(x,\cdot )\,d\eta _j \; \text { over } [0,T],\\ p(T)= p^0 \nabla g(x(T)). \end{array}\right. } \end{aligned}$$

From the Duhamel formula for backward linear Cauchy–Stieltjes problems (see Proposition A.6) and using notations introduced in Sect. A.3, it holds that

$$\begin{aligned} p(t)= \Phi (T,t)^\top \times \Big ( p^0 \nabla g(x(T)) \Big ) - \int _t^T \Phi (\tau ,t)^\top \times \partial _1 h(x(\tau ),\tau )^\top \times d\eta (\tau ), \end{aligned}$$

for all $t\in [0,T]$, where $\Phi (\cdot ,\cdot ):[0,T]^2 \rightarrow {\mathbb {R}}^{n \times n}$ stands for the state-transition matrix associated with $\partial _1 f(x,u,\cdot )\in \mathrm {L}^\infty ([0,T],{\mathbb {R}}^{n \times n})$.

Nonpositive averaged Hamiltonian gradient condition. From Inequality (9) and using notations introduced in Sect. A.3, it holds that

$$\begin{aligned} \lambda \Big \langle \nabla g(x(T)), w_z(T,u) \Big \rangle _{{\mathbb {R}}^n} + \int _0^T \Big \langle \partial _1 h(x(\tau ),\tau )\times w_z(\tau ,u), d\eta (\tau ) \Big \rangle \ge 0, \end{aligned}$$

for all $z \in \mathrm {PC}^{\mathbb {T}}_m$ with values in $\mathrm {U}$. From the definition of the variation vector $w_z(\cdot ,u)$ and the classical Duhamel formula for standard forward linear Cauchy problems, it holds that

$$\begin{aligned} w_z(\tau ,u) = \int _0^\tau \Phi (\tau ,s) \times \partial _2 f(x(s),u(s),s) \times (z(s)-u(s))\,ds, \end{aligned}$$

for all $\tau \in [0,T]$. Substituting this expression into the previous inequality and using the last Fubini formula given in Sect. A.3, it follows that

$$\begin{aligned}&\int _0^T \left\langle \Phi (T,s)^\top \times \Big (p^0 \nabla g(x(T)) \Big ), \partial _2 f(x(s),u(s),s)\times (z(s)-u(s)) \right\rangle _{{\mathbb {R}}^n}\,ds \\&\quad - \int _0^T \Big \langle \partial _2 f(x(s),u(s),s)\times (z(s)-u(s)), \\&\quad \int _s^T \Phi (\tau ,s)^\top \times \partial _1 h(x(\tau ),\tau )^\top \times d\eta (\tau ) \Big \rangle _{{\mathbb {R}}^n} \, ds \le 0, \end{aligned}$$

for all $z \in \mathrm {PC}^{\mathbb {T}}_m$ with values in $\mathrm {U}$. Finally, grouping like terms, we exactly obtain

$$\begin{aligned} \int _{0}^{T} \Big \langle p(s), \partial _2 f(x(s),u(s),s) \times (z(s)-u(s)) \Big \rangle _{{\mathbb {R}}^n} \,ds \le 0, \end{aligned}$$

for all $z\in \mathrm {PC}^{\mathbb {T}}_m$ with values in $\mathrm {U}$. For all $i=0,\ldots ,N-1$ and all $v \in \mathrm {U}$, let us consider $z_{i,v} \in \mathrm {PC}^{\mathbb {T}}_m$ with values in $\mathrm {U}$ as

$$\begin{aligned} z_{i,v}(s):=\left\{ \begin{array}{lcl} v &{} \text {if} &{} s \in [t_i,t_{i+1}),\\ u(s) &{} \text {if} &{} s \notin [t_i,t_{i+1}), \end{array} \right. \end{aligned}$$

for all $s \in [0,T]$. Substituting z by $z_{i,v}$ in the above inequality and from the definition of the Hamiltonian H, we exactly get that

$$\begin{aligned} \left\langle \int _{t_i}^{t_{i+1}} \partial _2 H(x(s),u_i,p(s),p^0,s) \,ds , v - u_i \right\rangle _{{\mathbb {R}}^m} \le 0, \end{aligned}$$

for all $v \in \mathrm {U}$ and all $i=0,\ldots ,N-1$. The proof of Theorem 3.1 is complete (in the case $L=0$).

1.2 The case $L \ne 0$

In the previous section we have proved Theorem 3.1 in the case $L = 0$ (without Lagrange cost). This section is dedicated to the case $L \ne 0$. Let $(x,u)\in \mathrm {AC}_n\times \mathrm {PC}^{\mathbb {T}}_m$ be a solution to Problem ($\mathrm {OSCP}$). Let us introduce

$$\begin{aligned} X(t) := \int _0^t L(x(\tau ),u(\tau ),\tau ) \, d\tau , \end{aligned}$$

for all $t \in [0,T]$. We see that the augmented couple $((x,X),u)\in \mathrm {AC}_{n+1}\times \mathrm {PC}^{\mathbb {T}}_m$ is a solution to the augmented optimal sampled-data control problem with running inequality state constraints of Mayer form given by

where ${\tilde{g}}: {\mathbb {R}}^{n+1} \rightarrow {\mathbb {R}}$ is defined by ${\tilde{g}}(x_1,X_1):=g(x_1)+X_1$ for all $(x_1,X_1)\in {\mathbb {R}}^{n+1}$ and where ${\tilde{h}} : {\mathbb {R}}^{n+1} \times [0,T] \rightarrow {\mathbb {R}}^q$ is defined by ${\tilde{h}}((x_1, X_1),t) := h(x_1,t)$ for all $(x_1,X_1)\in {\mathbb {R}}^{n+1}$ and all $t\in [0,T]$. Note that Problem ($\mathrm {OSCP}_{aug}$) satisfies all of the assumptions of Theorem 3.1 and is without Lagrange cost. We introduce the augmented Hamiltonian ${\tilde{H}} : {\mathbb {R}}^{n+1} \times {\mathbb {R}}^m \times {\mathbb {R}}^{n+1} \times [0,T] \rightarrow {\mathbb {R}}$ defined as

$$\begin{aligned} {\tilde{H}}((x,X),u,(p,P),t):= \left\langle \left( \begin{array}{c} p \\ P \end{array} \right) ,\left( \begin{array}{c} f(x,u,t) \\ L(x,u,t) \end{array} \right) \right\rangle _{{\mathbb {R}}^{n+1}}, \end{aligned}$$

for all $((x,X),u,(p,P),t)\in {\mathbb {R}}^{n+1} \times {\mathbb {R}}^m \times {\mathbb {R}}^{n+1} \times [0,T]$. Applying Theorem 3.1 (without Lagrange cost, proved in the previous section), we deduce the existence of a nontrivial couple $(p^0,\eta )$, where $p^0\le 0$ and $\eta =(\eta _j)_{j=1,\ldots ,q}\in \mathrm {NBV}_q$, such that all conclusions of Theorem 3.1 are satisfied. In particular, the adjoint vector $(p,P) \in \mathrm {BV}_{n+1}$ satisfies the backward linear Cauchy-Stieltjes problem given by

$$\begin{aligned} {\left\{ \begin{array}{ll} -\left( \begin{array}{c} dp \\ dP \end{array} \right) =\partial _{(x,X)} \left( \begin{array}{c} f \\ L \end{array} \right) (x,u,\cdot )^\top \times \left( \begin{array}{c} p \\ P \end{array} \right) \,dt - \sum _{j=1}^{q} \partial _{(x,X)} {\tilde{h}}_j((x, X),\cdot )\,d\eta _j \quad \text { over } [0,T], \\ \left( \begin{array}{c} p \\ P \end{array} \right) (T)=p^0 \nabla _{(x,X)} {\tilde{g}}((x,X)(T)). \end{array}\right. } \end{aligned}$$

We deduce that $P(T)=p^0$ and $dP=0$ over [0, T]. Thus $P(t)=p^0$ for all $t \in [0,T]$, and we obtain that $p \in \mathrm {BV}_n$ satisfies the backward linear Cauchy–Stieltjes problem

$$\begin{aligned} {\left\{ \begin{array}{ll} -dp= \Big ( \partial _1 f(x,u,\cdot )^\top \times p+ p^0 \partial _1 L(x,u,\cdot ) \Big ) \, dt - \sum _{j=1}^{q} \partial _1 h_j(x,\cdot ) \, d\eta _j \quad \text { over } [0,T], \\ p(T)=p^0 \nabla g(x(T)). \end{array}\right. } \end{aligned}$$

The rest of the proof is straightforward from all the necessary conditions obtained from the version of Theorem 3.1 without Lagrange cost.