Representation of Hamilton–Jacobi Equation in Optimal Control Theory with Unbounded Control Set

Abstract

In this paper, we study the existence of sufficiently regular representations of Hamilton–Jacobi equations in the optimal control theory with unbounded control set. We use a new method to construct representations for a wide class of Hamiltonians. This class is wider than any constructed before, because we do not require Legendre–Fenchel conjugates of Hamiltonians to be bounded. However, in this case we obtain representations with unbounded control set. We apply the obtained results to study regularities of value functions and correlations between variational and optimal control problems.

1 Introduction

The Hamilton–Jacobi equation (6) with a convex Hamiltonian H in the gradient variable can be studied with connection to calculus of variations problems, namely the value function of the calculus of variations problem given by (7) is the unique viscosity solution, see, e.g., [1,2,3,4,5,6,7]. The Hamilton–Jacobi equation (6) can be also studied with connection to optimal control problems. It is possible provided that there exists a sufficiently regular triple (A, f, l) satisfying the equality (4). In particular, the value function of the optimal control problem given by (8) is the unique viscosity solution, see, e.g., [1, 3, 4, 8, 9].

The triple (A, f, l), which satisfies the equality (4), is called a representation of the Hamiltonian H. In general, if a representation of H exists, then infinitely many other representations exist. There are also irregular representations among them. The triple (A, f, l), which satisfies the equality (4) and inherits Lipschitz-type properties of H, is called a faithful representation of H.

In this paper, we provide further developments of representation theorems from [10]. Misztela [10] studied faithful representations of Hamiltonians with the compact control set. A necessary condition for the existence of such representations is boundedness of Legendre–Fenchel conjugates of Hamiltonians on effective domains, see [10, Theorem 3.1]. However, in many cases Hamiltonians do not have bounded Legendre–Fenchel conjugates on effective domains. In Sect. 4, we see that for this type of Hamiltonians there exist faithful representations with the unbounded control set. We used a new method to construct a faithful representation. Our representation (A, f, l) of H is an epigraphical representation introduced in [10]. The construction of this representation is as follows: First, using Steiner selection we parameterize the set-valued map obtained via epigraph of the Legendre–Fenchel conjugate of H. Steiner selection guarantees that this parametrization $\mathsf {e}$ with the parameter set A is locally Lipschitz continuous with respect to the state variable. Next, we define the functions f and l as components of the function $\mathsf {e}$, i.e., $\mathsf {e}=(f,l)$. In view of [10, Proposition 5.7], any triple (A, f, l) obtained in such a way is a representation of H. Earlier, Frankowska–Sedrakyan [11] and Rampazzo [12] used a graphical representation to construct a faithful representation. In a graphical representation, the function l, without additional assumptions, may be discontinuous with respect to the state-control variable, see Sect. 3. Another differences between graphical and epigraphical representations can be found in [10]. Earlier, Ishii [13] proposed a representation involving continuous functions f and l with the infinite-dimensional control set A. The lack of local Lipschitz continuity of f and l with respect to the state variable and finite-dimensional control set A in Ishii [13] paper causes troubles in applications.

We present differences between representations with unbounded and compact control sets. The fact that a control set is not compact makes significant problems in applications which we discuss below. Therefore, compactness of a control set must be replaced by another property that is convenient in practice. The property (A3) from Theorem 4.1 which is a consequence of our construction of a faithful representation plays a role of such extra-property. Our extra-property is apparently new. In the literature, one usually requires coercivity of the function $l(t,x,\cdot )$, see, e.g., [9, Condition ($A_4$)]. However, the function $l(t,x,\cdot )$ from our faithful representation (A, f, l) does not have this property. Coercivity of the function $l(t,x,\cdot )$ enables us to study not only measurability of controls but also their integrability. In this paper, the extra-property plays a similar role, see Remarks 4.1 and 4.2. It is well-known that in applications one requires at least integrability of controls. In the case when the control set is compact the above problem does not occur, because every measurable control with values in the compact control set is integrable.

In general, the value functions (7) and (8) are not equal. However, in our case these value functions are identical due to the extra-property, see Corollary 4.1. Moreover, we obtain a fundamental relation between variational and optimal control problems, see Theorem 4.4. More precisely, we consider a variational problem associated with the given Lagrangian L. We define Hamiltonian H as the Legendre–Fenchel conjugate of L in its velocity variable. Applying our result to Hamiltonian H, we obtain its faithful representation (A, f, l). Then the variational problem associated with Lagrangian L is equivalent to the optimal control problem associated with the triple (A, f, l). Earlier, Olech [14] and Rockafeller [15, 16] investigated the opposite problem to ours.

Our faithful representations are stable, see Theorems 4.2 and 4.3. This fact is used in the proof of stability of value functions, see Sect. 6. The method of this proof is not standard, because properties of a faithful representation are nonstandard. These nonstandard properties are: an unbounded control set, the extra-property and the sublinear growth of l with respect to the control variable. In this case, one cannot apply methods from Sedrakyan [17] to prove stability of value functions. Indeed, this method uses compactness of the control set and boundedness of l independent of the control variable. We also prove that the value function is locally Lipschitz continuous, provided that the final cost function is locally Lipschitz continuous. In the proof of this fact, nonstandard boundedness of the function l plays a significant role.

The outline of the paper is as follows. Section 2 contains hypotheses and background material. In Sect. 3, we show differences between graphical and epigraphical representations with the unbounded control set. In Sect. 4, we gathered our main results. Sections 5 and 6 contain proofs.

2 Hypotheses and Background Material

We will need hypotheses and results similar to those in [10, Sect. 2].

(H1):

$H:[0,T]\times {\mathbb R}^{n}\times {\mathbb R}^{n}\rightarrow {\mathbb R}$ is Lebesgue measurable in t for any $x,p\in {\mathbb R}^n$;

(H2):

H(t, x, p) is continuous with respect to (x, p) for every $t\in [0,T]$;

(H3):

H(t, x, p) is convex with respect to p for every $(t,x)\in [0,T]\times {\mathbb R}^n$;

(H4):

There exists a measurable map $c:[0,T]\rightarrow [0,\infty [$ such that for every $t\in [0,T]$, $x,p,q\in {\mathbb R}^n$ one has $|H(t,x,p)-H(t,x,q)|\le c(t)(1+|x|)|p-q|$.

Let $\varphi$ be an extended-real-valued function from ${\mathbb R}^m$ to ${\mathbb R}\cup \{\pm \infty \}$. The sets: ${\mathrm {dom}}\,\varphi =\{z\in {\mathbb R}^m:\varphi (z)\not =\pm \infty \}$, ${\mathrm {gph}}\,\varphi =\{(z,r)\in {\mathbb R}^m\times {\mathbb R}:\varphi (z)=r\}$ and ${\mathrm {epi}}\,\varphi =\{(z,r)\in {\mathbb R}^m\times {\mathbb R}:\varphi (z)\le r\}$ are called the effective domain, the graph and the epigraph of $\varphi$, respectively. We say that $\varphi$ is proper if it never takes the value $-\infty$ and it is not identically equal to $+\infty$.

Let $H^{*}(t,x,\cdot )$ denotes the Legendre–Fenchel conjugate of $H(t,x,\cdot )$:

$$\begin{aligned} H^{*}(t,x,v)= \sup _{p\in {\mathbb R}^{n}}\,\{\,\langle v,p\rangle -H(t,x,p)\,\}. \end{aligned}$$

Using properties of the conjugate from [18], we can prove the following.

Proposition 2.1

Assume that H satisfies (H1)–(H3). Then

(C1):

$H^{*}:[0,T]\times {\mathbb R}^{n}\times {\mathbb R}^{n}\rightarrow {\mathbb R}\cup \{+\infty \}$ is Lebesgue-Borel-Borel measurable;

(C2):

$(x,v)\rightarrow H^{*}(t,x,v)$ is lower semicontinuous for every $t\in [0,T]$;

(C3):

$v\rightarrow H^{*}(t,x,v)$ is convex and proper for every $(t,x)\in [0,T]\times {\mathbb R}^n$;

(C4):

$\forall t\in [0,T]\;\forall x,\!v\in {\mathbb R}^n\;\forall x_i\rightarrow x \;\exists v_i\rightarrow v: H^{*}(t,x_i,v_i)\rightarrow H^{*}(t,x,v)$. Additionally, if H satisfies (H4), then

(C5):

$\forall (t,x,v)\in [0,T]\times {\mathbb R}^n\times {\mathbb R}^n: |v|>c(t)(1+|x|)\,\Rightarrow \, H^{*}(t,x,v) =+\infty$. Additionally, if H is continuous, then $H^{*}$ is lower semicontinuous and

(C6):

$\forall t\!\in \![0,T]\, x,\!v\!\in \!{\mathbb R}^n\;\forall (t_i,x_i)\!\rightarrow \! (t,x)\;\exists v_i\!\rightarrow \! v: H^{*}(t_i,x_i,v_i)\!\rightarrow \! H^{*}(t,x,v)$.

Let K be a nonempty subset of ${\mathbb R}^m$. We put $\Vert K\Vert :=\sup _{\xi \in K}|\xi |$. The distance from $y\in {\mathbb R}^m$ to K is defined by $d(y,K):=\inf _{\xi \in K}|y-\xi |$. A set-valued map $F:[0,T]\multimap {\mathbb R}^m$ is measurable if for each open set $U\subset {\mathbb R}^m$ the inverse image $F^{-1}(U):= \{t\in [0,T]: F(t)\cap U\not =\emptyset \}$ is Lebesgue measurable set. The set ${\mathrm {gph}}\,F:= \{(z,y): y\in F(z)\}$ is called a graph of the set-valued map F. A set-valued map $F:{\mathbb R}^n\multimap {\mathbb R}^m$ is lower semicontinuous in Kuratowski’s sense if for each open set $U\subset {\mathbb R}^m$ the set $F^{-1}(U)$ is open.

Let us define the set-valued map $E_{H^{*}}:[0,T]\times {\mathbb R}^n\multimap {\mathbb R}^n\times {\mathbb R}$ by the formula

$$\begin{aligned} E_{H^{*}}(t,x):={\mathrm {epi}}\,H^{*}(t,x,\cdot )= \{(v,\eta )\in {\mathbb R}^n\times {\mathbb R}: H^{*}(t,x,v)\le \eta \}. \end{aligned}$$

In view of Proposition 2.1 and Results in [18, Chap. 14], we obtain

Corollary 2.1

Assume that H satisfies (H1)–(H3). Then

(E1):

$(t,x)\rightarrow E_{H^{*}}(t,x)$ has a nonempty, closed, convex values;

(E2):

$x\rightarrow E_{H^{*}}(t,x)$ has a closed graph for every $t\in [0,T]$;

(E3):

$x\rightarrow E_{H^{*}}(t,x)$ is lower semicontinuous for every $t\in [0,T]$;

(E4):

$t\rightarrow E_{H^{*}}(t,x)$ is measurable for every $x\in {\mathbb R}^n$. Additionally, if H satisfies (H4), then

(E5):

$\Vert {\mathrm {dom}}\,{H^{*}}(t,x,\cdot )\Vert \le c(t)(1+|x|)$ for every $(t,x)\in [0,T]\times {\mathbb R}^n$. Additionally, if H is continuous, then

(E6):

$(t,x)\rightarrow E_{H^{*}}(t,x)$ has a closed graph and is lower semicontinuous.

Now we present Hausdorff continuity of a set-valued map $E_{H^{*}}$ in Hamiltonian and its conjugate terms. Let ${\mathbb B}(\bar{x},R)$ denote the closed ball in ${\mathbb R}^n$ of center $\bar{x}$ and radius $R\ge 0$. We put ${\mathbb B}_R:={\mathbb B}(0,R)$ and ${\mathbb B}:={\mathbb B}(0,1)$.

Theorem 2.1

See [10, Theorem 2.3] Assume that H satisfies (H1)–(H3). Then the following conditions are equivalent with the same map $k_R(\cdot )$:

(HLC):

For any $R>0$, there exists a measurable map $k_R:[0,T]\rightarrow [0,\infty [$ such that $|H(t,x,p)-H(t,y,p)|\le k_R(t)(1+|p|)|x-y|$ for all $t\in [0,T], x,y\in {\mathbb B}_R, p\in {\mathbb R}^n$.

(CLC):

For any $R>0$, there exists a measurable map $k_R:[0,T]\rightarrow [0,\infty [$ such that for all $t\in [0,T], x,y\in {\mathbb B}_R, v\in {\mathrm {dom}}\,H^{*}(t,x,\cdot )$ there exists $u\in {\mathrm {dom}}\,H^{*}(t,y,\cdot )$ satisfying the inequalities $|u-v|\le k_R(t)|y-x|$ and $H^{*}(t,y,u)\le H^{*}(t,x,v)+k_R(t)|y-x|$.

(ELC):

For any $R>0$, there exists a measurable map $k_R:[0,T]\rightarrow [0,\infty [$ such that $E_{H^{*}}(t,x)\subset E_{H^{*}}(t,y)+k_R(t)|x-y|({\mathbb B}\times [-1,1])$ for all $t\in [0,T]$, $x,y\in {\mathbb B}_R$.

For nonempty subsets K and D of ${\mathbb R}^m$, the extended Hausdorff distance between K and D is defined by the formula

$$\begin{aligned} \mathcal {H}(K,D):=\sup \{\,|d(x,K)-d(x,D)|: x\in {\mathbb R}^m\,\}\in {\mathbb R}\cup \{+\infty \}. \end{aligned}$$

By Theorem 2.1 (ELC), we obtain the following corollary.

Corollary 2.2

Assume that H satisfies (H1)–(H3) and (HLC). Then, we have $\mathcal {H}(E_{H^{*}}(t,x),E_{H^{*}}(t,y))\le 2k_R(t)|x-y|$ for all $t\in [0,T], x,y\in {\mathbb B}_R, R>0$.

3 Graphical and Epigraphical Representations of the Hamiltonian

A triple (A, f, l) is an epigraphical representation of H if

$$\begin{aligned} {\mathrm {gph}}\,H^{*}(t,x,\cdot )\subset (f(t,x,A),l(t,x,A))\subset {\mathrm {epi}}\,H^{*}(t,x,\cdot ), \end{aligned}$$

where $(f(t,x,A),l(t,x,A)):=\{(f(t,x,a),l(t,x,a)): a\in A\}$. A triple (A, f, l) is a graphical representation of H if $(f(t,x,A),l(t,x,A))={\mathrm {gph}}\,H^{*}(t,x,\cdot )$. We show differences between graphical and epigraphical representations of the Hamiltonian, whose conjugate is unbounded on the effective domain.

Let us define the Hamiltonian $\pmb {H}:{\mathbb R}\times {\mathbb R}\rightarrow {\mathbb R}$ by the formula

$$\begin{aligned} \pmb {H}(x,p):=\left\{ \begin{array}{ccl} (\sqrt{|xp|}-1)^2, &{} \text {if} &{} |xp|\,> 1, \\ 0, &{} \text {if} &{} |xp|\le 1. \end{array} \right. \end{aligned}$$

This Hamiltonian satisfies the conditions (H1)–(H4) and (HLC). Its conjugate $\pmb {H}^{*}:{\mathbb R}\times {\mathbb R}\rightarrow {\mathbb R}\cup \{+\infty \}$ has the following form

$$\begin{aligned} \pmb {H}^{*}(x,v)=\left\{ \begin{array}{ccl} +\infty , &{} \text {if} &{} |v|\ge |x|,\;x\not =0,\\ \frac{\displaystyle |v|}{\displaystyle |x|-|v|}, &{} \text {if} &{} |v|<|x|,\;x\not =0, \\ 0, &{} \text {if} &{} v=0,\; x=0,\\ +\infty , &{} \text {if} &{} v\not =0,\;x=0. \end{array} \right. \end{aligned}$$

The set ${\mathrm {dom}}\,\pmb {H}^{*}(x,\cdot )=]-|x|,|x|\,[$ is not closed and the function $v\rightarrow \pmb {H}^{*}(x,v)$ is not bounded on this set for every $x\in {\mathbb R}\setminus \{0\}$. Moreover, the function $\pmb {H}^{*}$ is not continuous on ${\mathrm {dom}}\,\pmb {H}^{*}$, because $\lim _{i\rightarrow \infty }\pmb {H}^{*}\left( 2/i,1/i\right) =1\not = 0=\pmb {H}^{*}(0,0)$.

Since $\pmb {H}$ satisfies (H1)–(H4) and (HLC), we can construct an epigraphical representation $({\mathbb R}\times {\mathbb R},f,l)$ of $\pmb {H}$ like in Theorem 4.1. However, the method of constructing a graphical representation given in [11, 12] cannot be applied to $\pmb {H}$, since the parametrization theorem of set-valued maps involves closed-valued maps. However, $x\rightarrow {\mathrm {dom}}\,\pmb {H}^{*}(x,\cdot )$ is not a closed-valued map. Therefore, we cannot utilize this approach to parametrize $x\rightarrow {\mathrm {dom}}\,\pmb {H}^{*}(x,\cdot )$. Nevertheless, to parametrize $x\rightarrow {\mathrm {dom}}\,\pmb {H}^{*}(x,\cdot )$ we can use an epigraphical representation $({\mathbb R}\times {\mathbb R},f,l)$ of $\pmb {H}$ from Theorem 4.1. Then $f(x,{\mathbb R}\times {\mathbb R})={\mathrm {dom}}\,\pmb {H}^{*}(x,\cdot )$. Let $\pmb {l}$ by given by

$$\begin{aligned} \pmb {l}(x,a_1,a_2)=\pmb {H}^{*}(x,f(x,a_1,a_2)). \end{aligned}$$

(1)

Of course, $({\mathbb R}\times {\mathbb R},f,\pmb {l})$ is a graphical representation of $\pmb {H}$. However, the function $\pmb {l}$ at the point (0, 0, r) is discontinuous for $r>0$. Indeed, let $a_1=r|x|/(1+r)$ and $a_2=r$ with $x\in {\mathbb R},r>0$. We observe that $(a_1,a_2)\in {\mathrm {epi}}\,\pmb {H}^{*}(x,\cdot )$. By the extra-property (A3) of Theorem 4.1, we have

$$\begin{aligned} f(x,a_1,a_2)=a_1\;\;\;\;\;\text {and}\;\;\;\;\; l(x,a_1,a_2)=a_2. \end{aligned}$$

(2)

Let $a_{1i}=r|x_i|/(1+r)$ and $a_{2i}=r$ with $x_i=1/i, r>0$. Then $(x_i,a_{1i},a_{2i})\rightarrow (0,0,r)$ as $i\rightarrow \infty$. By (2) we have $f(x_i,a_{1i},a_{2i})=a_{1i}$ and $f(0,0,r)=0$. By (1), we have $\pmb {l}(x_i,a_{1i},a_{2i})=\pmb {H}^{*}(x,a_{1i})=r$ and $\pmb {l}(0,0,r)=\pmb {H}^{*}(0,0)=0$. Suppose that $\pmb {l}$ is continuous. Then we get $r=\lim _{i\rightarrow \infty }\pmb {l}(x_i,a_{1i},a_{2i})=\pmb {l}(0,0,r)=0$. This contradicts the fact that $r>0$.

The lack of continuity of the function $\pmb {l}$ is not surprising, because the function $\pmb {l}$ is a composition of discontinuous and continuous functions. Such compositions usually are not continuous. However, it is not a rule. We observe that the function $\pmb {f}(x,a)=a|x|^2/(1+|a|\,|x|)$ is parametrization of ${\mathrm {dom}}\,\pmb {H}^{*}(x,\cdot )$ such that $\pmb {f}(x,{\mathbb R})={\mathrm {dom}}\,\pmb {H}^{*}(x,\cdot )$. Then, the function $\pmb {l}(x,a)=\pmb {H}^{*}(x,\pmb {f}(x,a))=|a|\,|x|$ is continuous. Therefore, $({\mathbb R},\pmb {f},\pmb {l})$ is a continuous graphical representation of $\pmb {H}$. In general, it is difficult to indicate the class of Hamiltonians with discontinuous conjugates for which continuous graphical representations with the unbounded control set exist.

We show that $\pmb {H}$ does not have a graphical representation $(\pmb {A},\pmb {f},\pmb {l})$ such that

$$\begin{aligned} |\pmb {l}(x,a)-\pmb {l}(y,a)|\le k_R\,|x-y|\;\;\; \text {for all}\;\;\, x,y\in {\mathbb B}_R,\; a\in \pmb {A}. \end{aligned}$$

(3)

In particular, the Hamiltonian $\pmb {H}$ does not have a graphical representation $(\pmb {A},\pmb {f},\pmb {l})$ which satisfies (A1) from Theorem 4.1. Assume by contradiction that the Hamiltonian $\pmb {H}$ has a graphical representation $(\pmb {A},\pmb {f},\pmb {l})$ satisfying (3). Let $x_i=R/i$ and $v_i=R/(2i)$ with $R>0$. Because of $v_i\in {\mathrm {dom}}\,\pmb {H}^{*}(x_i,\cdot )=\pmb {f}(x_i,\pmb {A})$, there exists $a_i\in \pmb {A}$ such that $\pmb {f}(x_i,a_i)=v_i$. We notice that $\pmb {l}(x_i,a_i)=\pmb {H}^{*}(x_i,\pmb {f}(x_i,a_i))=\pmb {H}^{*}(x_i,v_i)=1$ and $\pmb {l}(0,a_i)=\pmb {H}^{*}(0,\pmb {f}(0,a_i))=\pmb {H}^{*}(0,0)=0$. By (3), we get $1=|\pmb {l}(x_i,a_i)-\pmb {l}(0,a_i)|\le k_R\,|x_i|=k_R\,R/i$. Passing to the limit as $i\rightarrow \infty$ we get $1\le 0$, a contradiction.

4 Main Results

In this section, we describe the main results of the paper that concern faithful representations (A, f, l) with the unbounded control set $A:={\mathbb R}^{n}\times {\mathbb R}$.

Theorem 4.1

(Representation) Assume (H1)–(H4) and (HLC). Then there exist $f:[0,T]\times {\mathbb R}^n\times {\mathbb R}^{n+1}\rightarrow {\mathbb R}^n$ and $l:[0,T]\times {\mathbb R}^n\times {\mathbb R}^{n+1}\rightarrow {\mathbb R}$, measurable in t for all $(x,a)\in {\mathbb R}^n\times {\mathbb R}^{n+1}$ and continuous in (x, a) for all $t\in [0,T]$, such that for every $t\in [0,T], x,p\in {\mathbb R}^n$,

$$\begin{aligned} H(t,x,p)=\sup _{\;\;a\,\in \,{\mathbb R}^{n+1}}\,\{\,\langle \, p,f(t,x,a)\,\rangle -l(t,x,a)\,\} \end{aligned}$$

(4)

and $f(t,x,{\mathbb R}^{n+1})={\mathrm {dom}}\,H^{*}(t,x,\cdot )$. Moreover, we have the following.

(A1):

For any $R>0, t\in [0,T], x,y\in {\mathbb B}_R$, $a,b\in {\mathbb R}^{n+1}$

$$\begin{aligned} |f(t,x,a)-f(t,y,b)|\le & {} 10\,(n+1)\,(\,k_R(t)\,|x-y|+|a-b|\,),\\ |l(t,x,a)-l(t,y,b)|\le & {} 10\,(n+1)\,(\,k_R(t)\,|x-y|+|a-b|\,). \end{aligned}$$

(A2):

For any $t\in [0,T], x\in {\mathbb R}^n, a\in {\mathbb R}^{n+1}$

$$\begin{aligned} |f(t,x,a)|\le & {} c(t)(1+|x|)\;\;{ and}\;\;-|H(t,x,0)|\le l(t,x,a),\\ l(t,x,a)\le & {} 2|H(t,x,0)|+2c(t)(1+|x|)+3|a|. \end{aligned}$$

(A3):

$a=(f(t,x,a),l(t,x,a))$  for all  $a\in {\mathrm {epi}}\,H^{*}(t,x,\cdot ), t\in [0,T]$, $x\in {\mathbb R}^n$.

(A4):

Additionally, if H is continuous, so are f and l.

Property (A1) means that f and l are locally Lipschitz continuous in x with Lipschitz constants dependent on time and globally Lipschitz continuous in a with Lipschitz constant independent on time. Property (A2) implies that f has sublinear growth in x and l has sublinear growth in a. Property (A3) is called the extra-property. Property (A4) means that f and l are continuous if only H is continuous. The proof of Theorem 4.1 is given in Sect. 5.

Remark 4.1

We consider the representation $({\mathbb R}^{n+1}\!\!,f,l)$ of H defined as in Theorem 4.1. Then, by [10, Lemma 4.1], $\mathsf {e}(t,x,{\mathbb R}^{n+1})\subset {\mathrm {epi}}\,H^{*}(t,x,\cdot )$, where $\mathsf {e}=(f,l)$, and, by the extra-property, ${\mathrm {epi}}\,H^{*}(t,x,\cdot )=\mathsf {e}(t,x,{\mathrm {epi}}\,H^{*}(t,x,\cdot ))$. Hence, $\mathsf {e}(t,x,{\mathbb R}^{n+1})={\mathrm {epi}}\,H^{*}(t,x,\cdot )$. Thus, the extra-property implies that $({\mathbb R}^{n+1}\!\!,f,l)$ is an epigraphical representation of H. It turns out that the extra-property is much stronger than $\mathsf {e}(t,x,{\mathbb R}^{n+1})={\mathrm {epi}}\,H^{*}(t,x,\cdot )$. Indeed, we consider the absolutely continuous function $(x,u):[0,T]\rightarrow {\mathbb R}^{n+1}$ such that

$$\begin{aligned} (\dot{x},\dot{u})(t)\in {\mathrm {epi}}\,H^{*}(t,x(t),\cdot )\;\;\text {a.e.}\;\;t\in [0,T]. \end{aligned}$$

Then, by Filippov theorem and $\mathsf {e}(t,x,{\mathbb R}^{n+1})={\mathrm {epi}}\,H^{*}(t,x,\cdot )$, there exists a measurable control $\hat{a}(\cdot )$ defined on [0, T] with values in ${\mathbb R}^{n+1}$ such that

$$\begin{aligned} (\dot{x},\dot{u})(t)=\mathsf {e}(t,x(t),\hat{a}(t))\;\; \text {a.e.}\;\; t\in [0,T]. \end{aligned}$$

Obviously, the measurable control $\hat{a}(\cdot )$ may be not integrable. Whereas the extra-property with the control $\check{a}(\cdot ):=(\dot{x}(\cdot ),\dot{u}(\cdot ))$ implies that

$$\begin{aligned} (\dot{x},\dot{u})(t)=\mathsf {e}(t,x(t),\check{a}(t))\;\; \text {a.e.}\;\; t\in [0,T]. \end{aligned}$$

Since $(x,u)(\cdot )$ is an absolutely continuous function, $(\dot{x},\dot{u})(\cdot )$ is an integrable function. Thus, $\check{a}(\cdot )$ is also integrable. Besides, by Theorem 4.1 (A2),

$$\begin{aligned} |l(t,x(t),a(t))|\le 2\omega (t,x(t))+3|a(t)|\;\; \text {for all}\;\; t\in [0,T], \end{aligned}$$

(5)

where $\omega (t,x):=|H(t,x,0)|+c(t)(1+|x|)$. We observe that if $\omega (\cdot ,x(\cdot ))$ and $a(\cdot )$ are integrable functions, then the function $l(\cdot ,x(\cdot ),a(\cdot ))$ is integrable. However, if $a(\cdot )$ is measurable, then $l(\cdot ,x(\cdot ),a(\cdot ))$ may be not integrable.

Theorem 4.2

Let $H_i,H$, $i\in {\mathbb N}$, be continuous and satisfy (H1)–(H4), (HLC). We consider the representations $({\mathbb R}^{n+1}\!\!,f_i,l_i)$ and $({\mathbb R}^{n+1}\!\!,f,l)$ of $H_i$ and H, respectively, defined as in the proof of Theorem 4.1. If $H_i$ converge uniformly on compacts to H, then $f_i$ converge to f and $l_i$ converge to l uniformly on compacts in $[0,T]\times {\mathbb R}^n\times {\mathbb R}^{n+1}$.

Theorem 4.3

Let $H_i,H$, $i\in {\mathbb N}$, satisfy (H1)–(H4), (HLC). We consider the representations $({\mathbb R}^{n+1}\!\!,f_i,l_i)$ and $({\mathbb R}^{n+1}\!\!,f,l)$ of $H_i$ and H, respectively, defined as in the proof of Theorem 4.1. If $H_i(t,\cdot ,\cdot )$ converge uniformly on compacts to $H(t,\cdot ,\cdot )$ for every $t\in [0,T]$, then $f_i(t,\cdot ,\cdot )$ converge to $f(t,\cdot ,\cdot )$ and $l_i(t,\cdot ,\cdot )$ converge to $l(t,\cdot ,\cdot )$ uniformly on compacts in ${\mathbb R}^n\times {\mathbb R}^{n+1}$ for every $t\in [0,T]$.

4.1 Correlation Between Variational and Optimal Control Problems

In this subsection, we consider a special kind of variational and optimal control problems describing solutions of the Hamilton–Jacobi equation

$$\begin{aligned} \begin{array}{rll} -V_{t}+ H(t,x,-V_{x})=0 &{}\;\text {in}\;&{} ]0,T[\times {\mathbb R}^n, \\ V(T,x)=g(x) &{}\;\text {in}\;&{}{\mathbb R}^n, \end{array} \end{aligned}$$

(6)

with Hamiltonian H, which satisfies (H1)–(H4) and (HLC). These problems are theoretical in nature. Nevertheless, they can be useful in investigating practical problems. For instance, using these variational and optimal control problems we prove stability of value functions and local Lipschitz continuity of the value function. We consider the following variational problem

and the following optimal control problem

where $\mathcal {A}([t_0,T],{\mathbb R}^n)$ denotes the space of absolutely continuous functions.

Theorem 4.4

Assume that (H1)–(H4) and (HLC) hold with integrable functions $c(\cdot )$, $k_R(\cdot )$, $H(\cdot ,0,0)$. We consider the representation $({\mathbb R}^{n+1}\!\!,f,l)$ of H defined as in Theorem 4.1. Assume further that $\phi$ is a proper, lower semicontinuous function and there exists $M\ge 0$ such that $\min \{\,|z|,|x|\,\}\le M$ for all $(z,x)\in {\mathrm {dom}}\,\phi$. Then we have

$$\begin{aligned} \min \Gamma [x(\cdot )]\,=\,\min \Lambda [(x,a)(\cdot )]. \end{aligned}$$

Besides, if $\bar{x}(\cdot )$ is the optimal arc of $\mathcal {P}_{v}$ such that $\bar{x}(\cdot )\in {\mathrm {dom}}\,\Gamma$, then $(\bar{x},\bar{a})(\cdot )$ is the optimal arc of $\mathcal {P}_{c}$ with $\bar{a}(\cdot )=(\dot{\bar{x}}(\cdot ),H^{*}(\cdot ,\bar{x}(\cdot ),\dot{\bar{x}}(\cdot )))$ such that $(\bar{x},\bar{a})(\cdot )\in {\mathrm {dom}}\,\Lambda$. Conversely, if $(\bar{x},\bar{a})(\cdot )$ is the optimal arc of $\mathcal {P}_{c}$, then $\bar{x}(\cdot )$ is the optimal arc of $\mathcal {P}_{v}$.

The indicator function $\psi _K(\cdot )$ of the set K has value 0 on this set and $+\infty$ outside. Let $S _f(t_0,x_0)$ denotes the set of all trajectory-control pairs $(x,a)(\cdot )$ of the control system: $\dot{x}(t)=f(t,x(t),a(t))$ a.e. $t\in [t_0,T]$ and $x(t_0)=x_0$. Applying Theorem 4.4 to $\phi (z,x):=\psi _{\{x_0\}}(z)+g(x)$, we obtain:

Corollary 4.1

Assume that (H1)–(H4) and (HLC) hold with integrable functions $c(\cdot )$, $k_R(\cdot )$, $H(\cdot ,0,0)$. We consider the representation $({\mathbb R}^{n+1}\!\!,f,l)$ of H defined as in Theorem 4.1. Let g be a proper, lower semicontinuous function. Then, for all $(t_0,x_0)\in [0,T]\times {\mathbb R}^n$,

$$\begin{aligned} V(t_0,x_0)= & {} \min _{\begin{array}{c} \scriptstyle x(\cdot )\,\in \,\mathcal {A}([t_0,T],{\mathbb R}^n)\\ \scriptstyle x(t_0)=x_0 \end{array}}\!\!\big \{\,g(x(T))+\int _{t_0}^TH^{*}(t,x(t),\dot{x}(t))\,\mathrm{d}t\,\big \}\end{aligned}$$

(7)

$$\begin{aligned}= & {} \min _{(x,a)(\cdot )\,\in \, S _f(t_0,x_0)}\,\big \{\,g(x(T))+\int _{t_0}^Tl(t,x(t),a(t))\,\mathrm{d}t\,\big \}. \end{aligned}$$

(8)

Remark 4.2

Observe that the considered optimal control problem $\mathcal {P}_{c}$ has integrable controls. Investigating integrable controls is possible due to argumentation contained in Remark 4.1, see Sect. 5.1. In addition, correlation between the optimal control $\bar{a}(\cdot )$ and the optimal trajectory $\bar{x}(\cdot )$ can be expressed by the simple formula $\bar{a}(\cdot )=(\dot{\bar{x}}(\cdot ),H^{*}(\cdot ,\bar{x}(\cdot ),\dot{\bar{x}}(\cdot )))$.

4.2 Stability of Value Functions

Definition 4.1

A sequence of functions $\{\varphi _i\}_{i\in {\mathbb N}}$ is said to epi-converge to function $\varphi$ (e-$\lim _{i\rightarrow \infty }\varphi _i=\varphi$ for short) if, for every point $z\in {\mathbb R}^m$,

(i):

$\liminf _{i\rightarrow \infty }\varphi _i(z_i)\ge \varphi (z)$ for every sequence $z_i\rightarrow z$,

(ii):

$\limsup _{i\rightarrow \infty }\varphi _i(z_i)\le \varphi (z)$ for some sequence $z_i\rightarrow z$.

Theorem 4.5

Let $H_i,H, i\in {\mathbb N}$, satisfy (H1)–(H4) and (HLC) with the same integrable functions $c(\cdot )$, $k_R(\cdot )$. Let $|H(t,0,0)|+|H_i(t,0,0)|\le \mu (t)$ for all $t\in [0,T]$, $i\in {\mathbb N}$ and some integrable function $\mu (\cdot )$. Let $({\mathbb R}^{n+1}\!\!,f_i,l_i)$ and $({\mathbb R}^{n+1}\!\!,f,l)$ be representations of $H_i$ and H, respectively, defined as in the proof of Theorem 4.1. Let $V_i$ and V be the value functions associated with $({\mathbb R}^{n+1}\!\!,f_i,l_i,g_i)$ and $({\mathbb R}^{n+1}\!\!,f,l,g)$, respectively, and $H_i(t,\cdot ,\cdot )$ converge uniformly on compacts to $H(t,\cdot ,\cdot )$ for all $t\in [0,T]$. Then the following properties hold.

(a):

If $g_i$, g are continuous functions and $g_i$ converge to g uniformly on compacts in ${\mathbb R}^n$, then $V_i$ converge uniformly on compacts to V in $[0,T]\times {\mathbb R}^n$.

(b):

If $g_i$ and g are proper, lower semicontinuous and e-$\lim _{i\rightarrow \infty }g_i=g$, then e-$\lim _{i\rightarrow \infty }V_i=V$.

Remark 4.3

Proofs of stability of value functions in paper [10] were omitted, because they base on to simple methods. However, standard tools cannot be applied to Theorem 4.5, because we consider the optimal arc $(x_i,a_i)(\cdot )$ of $V_i(x_{i0},a_{i0})$ for all $i\in {\mathbb N}$. Fix $t\in [0,T]$. Then one can prove that the sequence $\{x_i(t)\}$ is bounded in ${\mathbb R}^n$. However, the sequence $\{a_i(t)\}$ is, in general, not bounded in ${\mathbb R}^{n+1}$. This means that to the sequence $\{(x_i(t),a_i(t))\}$ one cannot apply Theorem 4.3, because this theorem works only on compact subsets of the set ${\mathbb R}^n\times {\mathbb R}^{n+1}$. Therefore, we decided to strengthen Theorem 4.3 to work on sets of the type ${\mathbb B}_R\times {\mathbb R}^{n+1}$. It can be done by assuming significantly stronger convergence of Hamiltonians than the one considered in this paper, see [19, Theorem 3.14]. However, the strengthened Theorem 4.3 turned out to be needless, because introducing the nonstandard method of the proof overcame the above problem, see Sect. 6.

4.3 Lipschitz Continuous/Continuous/Lower Semicontinuous of $\pmb {V}$

Theorem 4.6

Assume that (H1)–(H4) and (HLC) hold with integrable functions $c(\cdot )$, $k_R(\cdot )$, $H(\cdot ,0,0)$. We consider the representation $({\mathbb R}^{n+1}\!\!,f,l)$ of H defined as in Theorem 4.1. Let g be a locally Lipschitz function. Assume that V is the value function associated with $({\mathbb R}^{n+1}\!\!,f,l,g)$. Then for every $M>0$ there exist $\alpha _M(\cdot )\in \mathcal {A}([0,T],{\mathbb R})$ and $C_M>0$ such that

$$\begin{aligned} |V(t,x)-V(s,y)|\le |\alpha _M(t)-\alpha _M(s)|+C_M|x-y| \end{aligned}$$

(9)

for all $t,s\in [0,T]$, $x,y\in {\mathbb B}_M$. Additionally, the value function V is locally Lipschitz continuous on $[0,T]\times {\mathbb R}^n$, if $c(\cdot )$, $k_R(\cdot )$, H are continuous functions.

Remark 4.4

We consider the optimal arc $(x_\pi ,a_\pi )(\cdot )$ of $V(x_{\pi },a_{\pi })$ for all $\pi \in \Pi$. In the proof of Theorem 4.6, one requires equi-boundedness of the family $\{l(\cdot ,x_\pi (\cdot ),a_\pi (\cdot )):\pi \in \Pi \}$ by an integrable function. By (5), we obtain

$$\begin{aligned} |l(\cdot ,x_\pi (\cdot ),a_\pi (\cdot ))|\le 2\omega (\cdot ,x_\pi (\cdot ))+3|a_\pi (\cdot )|\;\;\; \text {for all}\;\; \;\pi \in \Pi . \end{aligned}$$

One can prove that the family $\{x_\pi (\cdot ):\pi \in \Pi \}$ is equi-bounded by a constant function. However, the family $\{a_\pi (\cdot ):\pi \in \Pi \}$ is not bounded in general. Whereas, if g is a locally Lipschitz function, then there exists an integrable function that equi-bounds the family $\{a_\pi (\cdot ):\pi \in \Pi \}$, see [20, Theorem 4.7]. Thus, the family $\{l(\cdot ,x_\pi (\cdot ),a_\pi (\cdot )):\pi \in \Pi \}$ can be bounded by an integrable function. It turns out that proceeding adequately in the proof of Theorem 4.6 we can omit equi-boundedness of optimal controls, see Sect. 6. In the literature, the above problem is solved assuming boundedness of the function l independent of a, see [8].

Theorem 4.7

Assume that (H1)–(H4), (HLC) hold with integrable functions $c(\cdot )$, $k_R(\cdot )$, $H(\cdot ,0,0)$. We consider the representation $({\mathbb R}^{n+1}\!\!,f,l)$ of H defined as in Theorem 4.1. Let g be a continuous/lower semicontinuous function. Then the value function associated with $({\mathbb R}^{n+1}\!\!\!,f,l,g)$ is continuous/lower semicontinuous on $[0,T]\times {\mathbb R}^n$.

Remark 4.5

Theorem 4.7 is a direct consequence of Theorems 6.1 and 6.3.

5 Representation, Optimality and Stability Theorems

The support function $\sigma (K,\cdot ):{\mathbb R}^m\rightarrow {\mathbb R}$ of a nonempty, convex, compact set $K\subset {\mathbb R}^m$ is a convex real-valued function defined by

$$\begin{aligned} \sigma (K,p):=\max _{x\in K}\,\langle p,x\rangle ,\quad \forall \,p\in {\mathbb R}^m. \end{aligned}$$

Let $\sum _{^{\, m-1}}$ denote the unit sphere in ${\mathbb R}^m$ and let $\mu$ be the measure on $\sum _{^{\, m-1}}$ proportional to the Lebesgue measure and satisfying $\mu (\sum _{^{\, m-1}})=1$.

Definition 5.1

Let $m\in {\mathbb N}\setminus \{1\}$. For any nonempty, convex, compact subset K of ${\mathbb R}^m$, its Steiner point is defined by

$$\begin{aligned} s_m(K):=m\int _{\sum _{^{\, m-1}}}p\,\sigma (K,p)\;\mu (dp). \end{aligned}$$

One can show that $s_m(\cdot )$ is a selection, i.e., $s_m(K)\in K$, cf. [21, p. 366].

Theorem 5.1

Let a set-valued map $E:[0,T]\times {\mathbb R}^n\multimap {\mathbb R}^m$ satisfy (E1)–(E4). Then there exists a single-valued map $\mathsf {e}:[0,T]\times {\mathbb R}^n\times {\mathbb R}^m\rightarrow {\mathbb R}^m$ such that $\mathsf {e}(\cdot ,x,a)$ is measurable for every $(x,a)\in {\mathbb R}^n\times {\mathbb R}^m$ and $\mathsf {e}(t,\cdot ,\cdot )$ is continuous for every $t\in [0,T]$. Moreover, we have the following.

$\pmb {(\text {a}_1)}$:

$\mathsf {e}(t,x,{\mathbb R}^m)=E(t,x)$ for all $t\in [0,T]$, $x\in {\mathbb R}^n$;

$\pmb {(\text {a}_2)}$:

$a=\mathsf {e}(t,x,a)$ for all $a\in E(t,x)$, $t\in [0,T]$, $x\in {\mathbb R}^n$;

$\pmb {(\text {a}_3)}$:

$|\mathsf {e}(t,x,a)|\le 3|a|+2d(0,E(t,x))$ for all $a\in {\mathbb R}^m$, $t\in [0,T]$, $x\in {\mathbb R}^n$;

$\pmb {(\text {a}_4)}$:

$|\mathsf {e}(t,x,a)-\mathsf {e}(t,y,b)|\le 5m[\mathcal {H}(E(t,x),E(t,y))+|a-b|]$ for all $t\in [0,T]$, $x,y\in {\mathbb R}^n$, $a,b\in {\mathbb R}^m$;

$\pmb {(\text {a}_5)}$:

Additionally, if (E6) is verified, then $\mathsf {e}$ is continuous.

Proof

Let $(t,x,a)\in [0,T]\times {\mathbb R}^n\times {\mathbb R}^m$. We define the set-valued map $\Phi$ by

$$\begin{aligned} \Phi (t,x,a):= E(t,x)\cap {\mathbb B}(a,2d(a,E(t,x))). \end{aligned}$$

We notice that the set-valued map $\Phi$ is defined as in the proof of Theorem 5.6 from [10] with $\omega \equiv 1$. Next, we define the single-valued map $\mathsf {e}$ by

$$\begin{aligned} \mathsf {e}(t,x,a):= s_m(\Phi (t,x,a)), \end{aligned}$$

where $s_m$ in the Steiner selection. Since $\Phi$ is defined as in the proof of Theorem 5.6 from [10], so the single-valued map $\mathsf {e}$ is well-defined. Moreover, $\mathsf {e}(\cdot ,x,a)$ is measurable for every $(x,a)\in {\mathbb R}^n\times {\mathbb R}^m$ and $\mathsf {e}(t,\cdot ,\cdot )$ is continuous for every $t\in [0,T]$. If we assume that $\omega \equiv 1$, then by [10, Theorem 5.6] and [10, Lemma 5.1] we obtain ($\text {a}_4$) and ($\text {a}_5$). It remains to prove ($\text {a}_1$)–($\text {a}_3$).

By Definition 5.1, we obtain that, for all $t\in [0,T]$, $x\in {\mathbb R}^n$, $a\in {\mathbb R}^m$,

$$\begin{aligned} \mathsf {e}(t,x,a)=s_m(\Phi (t,x,a))\in \Phi (t,x,a)=E(t,x)\cap {\mathbb B}(a,2d(a,E(t,x))). \end{aligned}$$

(10)

To prove ($\text {a}_2$), we observe that by (10) we get $|\mathsf {e}(t,x,a)-a|\le 2d(a,E(t,x))$ for all $t\in [0,T]$, $x\in {\mathbb R}^n$, $a\in {\mathbb R}^m$. Hence $a=\mathsf {e}(t,x,a)$ for all $a\in E(t,x)$, $t\in [0,T]$, $x\in {\mathbb R}^n$.

To prove ($\text {a}_1$), we observe that by (10) we get $\mathsf {e}(t,x,{\mathbb R}^m)\subset E(t,x)$ for all $(t,x)\in [0,T]\times {\mathbb R}^n$. The latter, together with ($\text {a}_2$), implies that $E(t,x)= \mathsf {e}(t,x,E(t,x))\subset \mathsf {e}(t,x,{\mathbb R}^m)\subset E(t,x)$ for all $(t,x)\in [0,T]\times {\mathbb R}^n$. This means that $\mathsf {e}(t,x,{\mathbb R}^m)=E(t,x)$ for all $(t,x)\in [0,T]\times {\mathbb R}^n$.

To prove ($\text {a}_3$), we observe that by (10) we get $|\mathsf {e}(t,x,a)-a|\le 2d(a,E(t,x))$ for all $t\in [0,T]$, $x\in {\mathbb R}^n$, $a\in {\mathbb R}^m$. The latter, together with the inequality $d(a,E(t,x))\le d(0,E(t,x))+|a|$, implies that $|\mathsf {e}(t,x,a)|\le 3|a|+2d(0,E(t,x))$ for all $a\in {\mathbb R}^m$, $t\in [0,T]$, $x\in {\mathbb R}^n$. $\square$

Theorem 5.2

Assume that H satisfies (H1)–(H4), (HLC). Then there exists a function $\mathsf {e}:[0,T]\times {\mathbb R}^n\times {\mathbb R}^{n+1}\rightarrow {\mathbb R}^{n+1}$ such that $\mathsf {e}(\cdot ,x,a)$ is measurable for every $(x,a)\in {\mathbb R}^n\times {\mathbb R}^{n+1}$ and $\mathsf {e}(t,\cdot ,\cdot )$ is continuous for every $t\in [0,T]$. Moreover, we have the following.

$\pmb {(\mathsf {e}_1)}$:

$\mathsf {e}(t,x,{\mathbb R}^{n+1})={\mathrm {epi}}\,H^{*}(t,x,\cdot )$ for all $t\in [0,T]$, $x\in {\mathbb R}^n$;

$\pmb {(\mathsf {e}_2)}$:

$a=\mathsf {e}(t,x,a)$ for all $a\in {\mathrm {epi}}\,H^{*}(t,x,\cdot )$, $t\in [0,T]$, $x\in {\mathbb R}^n$;

$\pmb {(\mathsf {e}_3)}$:

$|\mathsf {e}(t,x,a)|\le 2|H(t,x,0)|+2c(t)(1+|x|)+3|a|$ for all $t\in [0,T]$, $x\in {\mathbb R}^n$, $a\in {\mathbb R}^{n+1}$;

$\pmb {(\mathsf {e}_4)}$:

$|\mathsf {e}(t,x,a)-\mathsf {e}(t,y,b)|\le 10(n+1)(k_R(t)|x-y|+|a-b|)$ for all $t\in [0,T]$, $x,y\in {\mathbb B}_R$, $a,b\in {\mathbb R}^{n+1}$ and $R>0$;

$\pmb {(\mathsf {e}_5)}$:

Additionally, if H is continuous, so is $\mathsf {e}$.

Proof

Let $E(t,x):=E_{H^{*}}(t,x)={\mathrm {epi}}\,H^{*}(t,x,\cdot )$ for every $(t,x)\in [0,T]\times {\mathbb R}^n$. Because of Corollaries 2.1 and 2.2, the function E satisfies assumptions of Theorem 5.1. Therefore, there exists a map $\mathsf {e}:[0,T]\times {\mathbb R}^{n}\times {\mathbb R}^{n+1}\rightarrow {\mathbb R}^{n+1}$ such that $\mathsf {e}(\cdot ,x,a)$ is measurable for every $(x,a)\in {\mathbb R}^n\times {\mathbb R}^{n+1}$ and $\mathsf {e}(t,\cdot ,\cdot )$ is continuous for every $t\in [0,T]$. Moreover, the map $\mathsf {e}$ satisfies ($\text {a}_1$)–($\text {a}_5$) from Theorem 5.1. By Theorem 5.1 ($\text {a}_4$) and Corollary 2.2, we have

$$\begin{aligned} |\mathsf {e}(t,x,a)-\mathsf {e}(t,y,b)|\le & {} 5(n+1)[\mathcal {H}(E(t,x),E(t,y))+|a-b|]\\\le & {} 10(n+1)k_R(t)|x-y|+5(n+1)|a-b| \end{aligned}$$

for all $t\in [0,T]$, $x,y\in {\mathbb B}_R$, $a,b\in {\mathbb R}^{n+1}$, $R>0$. It means that ($\mathsf {e}_4$) is satisfied. Moreover, if we assume that H is continuous, then by Corollary 2.1 we get that (E6) is verified. Thus, by Theorem 5.1 ($\text {a}_5$), we obtain that the map $\mathsf {e}$ is continuous. We observe that ($\mathsf {e}_1$) and ($\mathsf {e}_2$) follow from definition of E and the properties ($\text {a}_1$) and ($\text {a}_2$) in Theorem 5.1. It remains to prove ($\mathsf {e}_3$).

Fix $(t,x)\in [0,T]\times {\mathbb R}^n$. By (C1)–(C5), there exists $\bar{v}\in {\mathrm {dom}}\,H^{*}(t,x,\cdot )$ such that $H(t,x,0)=H^{**}(t,x,0)=-H^{*}(t,x,\bar{v})$ and $|\bar{v}|\le c(t)(1+|x|)$. We see $(\bar{v},H^{*}(t,x,\bar{v}))\in E(t,x)$. The latter, together with Theorem 5.1 ($\text {a}_3$), implies

$$\begin{aligned} |\mathsf {e}(t,x,a)|\le & {} 3|a|+2d(0,E(t,x))\le 3|a|+2|(\bar{v},H^{*}(t,x,\bar{v}))|\\\le & {} 3|a|+2|\bar{v}|+2|H^{*}(t,x,\bar{v})|\le 3|a|+2c(t)(1+|x|)+2|H(t,x,0)|. \end{aligned}$$

This completes the proof of the theorem. $\square$

Remark 5.1

Let $\mathsf {e}:[0,T]\times {\mathbb R}^n\times {\mathbb R}^{n+1}\rightarrow {\mathbb R}^{n+1}$ be the function from Theorem 5.2. We define $f:[0,T]\times {\mathbb R}^n\times {\mathbb R}^{n+1}\rightarrow {\mathbb R}^n$ and $l:[0,T]\times {\mathbb R}^n\times {\mathbb R}^{n+1}\rightarrow {\mathbb R}$ by

$$\begin{aligned} f(t,x,a):=\pi _1(\mathsf {e}(t,x,a))\;\;\;\text {and}\;\;\;l(t,x,a):=\pi _2(\mathsf {e}(t,x,a)), \end{aligned}$$

where $\pi _1(v,\eta )=v$ and $\pi _2(v,\eta )=\eta$ for all $v\in {\mathbb R}^n$ and $\eta \in {\mathbb R}$. Then for all $t\in [0,T]$, $x\in {\mathbb R}^n$, $a\in {\mathbb R}^{n+1}$ the equality $\mathsf {e}(t,x,a)=(f(t,x,a),l(t,x,a))$ holds. Thus, for all $t\in [0,T]$, $x,y\in {\mathbb R}^n$, $a,b\in {\mathbb R}^{n+1}$, we get $|l(t,x,a)|\le |\mathsf {e}(t,x,a)|$,

$$\begin{aligned} |f(t,x,a)-f(t,y,b)|\le & {} |\mathsf {e}(t,x,a)-\mathsf {e}(t,y,b)|,\\ |l(t,x,a)-l(t,y,b)|\le & {} |\mathsf {e}(t,x,a)-\mathsf {e}(t,y,b)|. \end{aligned}$$

From the above inequalities, it follows that the properties of the function $\mathsf {e}$ are inherited by functions f and l. It is not difficult to show that Theorem 4.1 follows from [10, Proposition 5.7], Theorem 5.2 and Corollary 2.1 (E5).

5.1 The Optimality Theorem

The proof of Theorem 4.4 is similar to the proof of [10, Theorem 3.13], so we omit it. In this subsection, we describe only the differences in these proofs due to the extra-property. Let $\mathcal {I}_{f}([t_0,T],{\mathbb R}^{2n+1})$ [resp. $\mathcal {M}_{f}([t_0,T],{\mathbb R}^{2n+1})$] denote the set of all absolutely integrable [resp. absolutely measurable] pairs $(x,a)(\cdot )$ which satisfies $\dot{x}(t)=f(t,x(t),a(t))$ for a.e. $t\in [t_0,T]$. Analogously, as in [10, Sect. 7], we can show that the functionals $\Gamma [\cdot ]$ and $\Lambda [\cdot ]$ are well-defined and

$$\begin{aligned} -\infty <\min \left\{ \inf _{x\in \mathcal {A}([t_0,T],{\mathbb R}^n)}\Gamma [x],\inf _{(x,a)\in \mathcal {M}_{f}([t_0,T],{\mathbb R}^{2n+1})}\Lambda [(x,a)]\right\} . \end{aligned}$$

(11)

The differences in the proofs of Theorem 4.4 and [10, Theorem 3.13] are related to the following equalities:

$$\begin{aligned}&\inf _{x\in \mathcal {A}([t_0,T],{\mathbb R}^n)}\Gamma [x]=\inf _{(x,a)\in \mathcal {M}_{f}([t_0,T],{\mathbb R}^{2n+1})}\Lambda [(x,a)],\end{aligned}$$

(12)

$$\begin{aligned}&\inf _{x\in \mathcal {A}([t_0,T],{\mathbb R}^n)}\Gamma [x]=\inf _{(x,a)\in \mathcal {I}_{f}([t_0,T],{\mathbb R}^{2n+1})}\Lambda [(x,a)]. \end{aligned}$$

(13)

Using [10, Lemma 4.1], we can show that $\text {LS}(12)\le \text {RS}(12)$, see [10, Sect. 7]. The latter, together with $\mathcal {I}_{f}([t_0,T],{\mathbb R}^{2n+1})\subset \mathcal {M}_{f}([t_0,T],{\mathbb R}^{2n+1})$, implies that $\text {LS}(13)\le \text {RS}(13)$. The proofs of the opposite inequalities require an appropriate definition of control, see [10, Sect. 7]. Using Filippov theorem to define a measurable control, see Remark 4.1, we can show that $\text {LS}(12)\ge \text {RS}(12)$. Whereas, using the extra-property to define an integrable control, see Remark 4.1, we can show that $\text {LS}(13)\ge \text {RS}(13)$. Therefore, without the extra-property we could show only the equality (12). Whereas, having the extra-property we can prove both equalities (12) and (13).

From the equality (13) and its proof, it follows that if $\bar{x}(\cdot )$ is the optimal arc of $\mathcal {P}_{v}$ such that $\bar{x}(\cdot )\in {\mathrm {dom}}\,\Gamma$, then $(\bar{x},\bar{a})(\cdot )$ is the optimal arc of $\mathcal {P}_{c}$ with $\bar{a}(\cdot )=(\dot{\bar{x}}(\cdot ),H^{*}(\cdot ,\bar{x}(\cdot ),\dot{\bar{x}}(\cdot )))$ such that $(\bar{x},\bar{a})(\cdot )\in {\mathrm {dom}}\,\Lambda$; conversely, if $(\bar{x},\bar{a})(\cdot )$ is the optimal arc of $\mathcal {P}_{c}$, then $\bar{x}(\cdot )$ is the optimal arc of $\mathcal {P}_{v}$. Using the latter with [10, Theorem 7.6], we can replace “$\inf$” by “$\min$” in (12) and (13).

5.2 The Stability Theorems

The proofs of Theorems 4.2 and 4.3 are consequences of [10, Theorem 6.6 and Remark 6.7], if we assume that $\omega _i\equiv 1$ for all $i\in {\mathbb N}\cup \{0\}$ and $H_0=H$. In [10], one assumed that for all $i\in {\mathbb N}\cup \{0\}$ the function $\omega _i$ is given by

$$\begin{aligned} \omega _i(t,x)=|\lambda _i(t,x)|+|H_i(t,x,0)|+c_i(t)(1+|x|)+1\;\;\text {with}\;\;\;t\in [0,T],\;x\in {\mathbb R}^n, \end{aligned}$$

where $c_i$ is coefficient in (H4) and $\lambda _i$ is upper-boundedness of $H_i^{*}$. In [10, Theorem 6.6], convergence $\omega _i$ to $\omega _0$ is required. For this reason, in [10, Theorems 3.8 and 3.9] one assumes convergence $H_i$ to $H_0$ as well as convergence $\lambda _i$ to $\lambda _0$ and $c_i$ to $c_0$. Since, in our case $\omega _i\equiv 1$, so Theorems 4.2 and 4.3 do not need convergence $c_i$ to $c_0$.

6 Regularities of Value Functions

Given real numbers $\tau$ and $\nu$, we put $\tau \wedge \nu :=\min \{\,\tau ,\nu \,\}$ and $\tau \vee \nu :=\max \{\,\tau ,\nu \,\}$. Let $\Vert \cdot \Vert$ denote the supremum norm in $C([0,T],{\mathbb R}^{m})$ and $\Vert \cdot \Vert _{L^1}$ denote the standard norm in $L^1([0,T],{\mathbb R}^{m})$.

6.1 Upper Semicontinuity of Value Functions

Theorem 6.1

Let $({\mathbb R}^{n+1}\!\!,f_i,l_i)$ and $({\mathbb R}^{n+1}\!\!,f,l)$ be as in Theorem 4.5. Assume that $g_i$ and g are continuous functions and $g_i$ converge to g uniformly on compacts in ${\mathbb R}^n$. Let $V_i$ and V be the value functions associated with $({\mathbb R}^{n+1}\!\!,f_i,l_i,g_i)$ and $({\mathbb R}^{n+1}\!\!,f,l,g)$, respectively. Then, for every $(t_0,x_0)\in [0,T]\times {\mathbb R}^n$, we have

$$\begin{aligned} \limsup \nolimits _{i\rightarrow \infty }V_i(t_{i0},x_{i0})\le V(t_0,x_0)\;\;{ for every sequence}\;\; (t_{i0},x_{i0})\rightarrow (t_0,x_0). \end{aligned}$$

Proof

Let us fix $t_{i0},t_0\in [0,T]$, $x_{i0},x_0\in {\mathbb R}^n$ such that $(t_{i0},x_{i0})\rightarrow (t_0,x_0)$. Then there exists $M>0$ such that $x_{i0},x_0\in {\mathbb B}_M$. Without loss of generality, we may assume $V(t_0,x_0)<+\infty$ since otherwise there is nothing to prove. Then by Corollary 4.1, there exists the optimal arc $(\bar{x},\bar{a})(\cdot )$ of $V(t_0,x_0)$ defined on $[t_0,T]$. We extend $\bar{a}(\cdot )$ from $[t_0,T]$ to [0, T] by setting $\bar{a}(t)=0$ for $t\in [0,t_0]$. Next, because of the sublinear growth of f, we extend $\bar{x}(\cdot )$ from $[t_0,T]$ to [0, T] such that $(\bar{x},\bar{a})(\cdot )\in S _f(t_0,x_0)$. Now we choose $x_i(\cdot )$ defined on [0, T] such that $(x_i,\bar{a})(\cdot )\in S _{f_i}(t_{i0},x_{i0})$. Then, our assumptions and Gronwall’s Lemma imply

$$\begin{aligned} \Vert x_i\Vert \vee \Vert \bar{x}\Vert\le & {} \big (M+\Vert c\Vert _{L^1}\big )\exp \big (\Vert c\Vert _{L^1}\big )=:R, \end{aligned}$$

(14)

$$\begin{aligned} \Vert x_i-\bar{x}\Vert\le & {} \Big (|x_{i0}-x_0|+\Vert f_i[x_i]-f[x_i]\Vert _{L^1}+\!\!\int \limits _{\scriptscriptstyle t_{i0}\wedge t_0}^{\scriptscriptstyle t_{i0}\vee t_0}\omega _{\scriptscriptstyle R}[t]\,\mathrm{d}t\Big )D, \end{aligned}$$

(15)

where $\omega _{\scriptscriptstyle R}[\cdot ]:=2\mu (\cdot )+(10(n+1)k_R(\cdot )+2c(\cdot ))(1+R)$, $D:=\exp (\Vert \omega _{\scriptscriptstyle R}\Vert _{L^1})$, $f_{i}[x_i](\cdot ):=f_i(\cdot ,x_i(\cdot ),\bar{a}(\cdot ))$, $f[x_i](\cdot ):=f(\cdot ,x_i(\cdot ),\bar{a}(\cdot ))$. We notice that

$$\begin{aligned} \Vert f_i[x_i]-f[x_i]\Vert _{L^1}\le \Vert \Psi _i\Vert _{L^1}, \end{aligned}$$

(16)

where $\Psi _i(\cdot ):=\sup \nolimits _{z\in {\mathbb B}_R}|\,f_i(\cdot ,z,\bar{a}(\cdot ))-f(\cdot ,z,\bar{a}(\cdot ))\,|$. By Theorem 4.1 (A2) we get $\Psi _i(t)\le 2c(t)(1+R)$ for all $t\in [0,T]$. Since $f_i(t,\cdot ,\bar{a}(t))$ converge to $f(t,\cdot ,\bar{a}(t))$ uniformly on compacts in ${\mathbb R}^n$ for all $t\in [0,T]$, we have $\lim _{i\rightarrow \infty }\Psi _i(t)=0$ for all $t\in [0,T]$. Therefore, by virtue of Lebesgue’s theorem and (16), we obtain

$$\begin{aligned} \lim \nolimits _{i\rightarrow \infty }\Vert f_i[x_i]-f[x_i]\Vert _{L^1}=0. \end{aligned}$$

(17)

Observe that (15), together with (17), implies $\lim _{i\rightarrow \infty }\Vert x_i-\bar{x}\Vert =0$. Since $l_i(t,\cdot ,\bar{a}(t))$ and $l(t,\cdot ,\bar{a}(t))$ are continuous, $l_i(t,\cdot ,\bar{a}(t))$ converge to $l(t,\cdot ,\bar{a}(t))$ uniformly on compacts in ${\mathbb R}^n$ and $x_i(t)\rightarrow \bar{x}(t)$ for all $t\in [0,T]$, we have $l_i(t,x_i(t),\bar{a}(t))\rightarrow l(t,\bar{x}(t),\bar{a}(t))$ for all $t\in [0,T]$. By Theorem 4.1 (A2) we get $|l_i(t,x_i(t),\bar{a}(t))|\le \omega _R[t]+3|\bar{a}(t)|$ for all $t\in [0,T]$. Therefore,

$$\begin{aligned} \lim \nolimits _{i\rightarrow \infty }\Vert l_i[x_i]-l[\bar{x}]\Vert _{L^1}=0. \end{aligned}$$

(18)

Again by our assumptions and Gronwall’s Lemma, we obtain

$$\begin{aligned}&V(t_0,x_0)-g(\bar{x}(T))=\int \limits _{\scriptscriptstyle t_0}^{\scriptscriptstyle T}l[\bar{x}](t)\,\mathrm{d}t\nonumber \\&\quad \ge \int \limits _{\scriptscriptstyle t_{i0}}^{\scriptscriptstyle T}l_{i}[x_i](t)\,\mathrm{d}t-\Vert l_i[x_i]-l[\bar{x}]\Vert _{L^1}- \!\!\!\int \limits _{\scriptscriptstyle t_{i0}\wedge t_0}^{\scriptscriptstyle t_{i0}\vee t_0}\!\Big (\omega _{\scriptscriptstyle R}[t]+3|\bar{a}(t)|\Big )\mathrm{d}t\nonumber \\&\quad \ge V_i(t_{i0},x_{i0})-g_i(x_i(T))-\Vert l_i[x_i]-l[\bar{x}]\Vert _{L^1}- \!\!\!\int \limits _{\scriptscriptstyle t_{i0}\wedge t_0}^{\scriptscriptstyle t_{i0}\vee t_0}\!\Big (\omega _{\scriptscriptstyle R}[t]+3|\bar{a}(t)|\Big )\mathrm{d}t,\nonumber \\ \end{aligned}$$

(19)

where $\,l_{i}[x_i](\cdot ):=l_i(\cdot ,x_i(\cdot ),\bar{a}(\cdot ))\,$ and $\,l[\bar{x}](\cdot ):=l(\cdot ,\bar{x}(\cdot ),\bar{a}(\cdot ))$. Since $g_i$ and g are continuous functions, $g_i$ converge to g uniformly on compacts in ${\mathbb R}^n$, and $x_i(T)\rightarrow \bar{x}(T)$, we see that $g_i(x_i(T))\rightarrow g(\bar{x}(T))$. The latter, together with (18) and (19), imply that $\limsup _{i\rightarrow \infty }V_i(t_{i0},x_{i0})\le V(t_0,x_0)$. $\square$

Theorem 6.2

Let $({\mathbb R}^{n+1}\!\!,f_i,l_i)$ and $({\mathbb R}^{n+1}\!\!,f,l)$ be as in Theorem 4.5. Assume that $g_i$ and g are proper, lower semicontinuous and e-$\lim _{i\rightarrow \infty }g_i=g$. Let $V_i$ and V be the value functions associated with $({\mathbb R}^{n+1}\!\!,f_i,l_i,g_i)$ and $({\mathbb R}^{n+1}\!\!,f,l,g)$, respectively. Then, for every $(t_0,x_0)\in [0,T]\times {\mathbb R}^n$, we have

$$\begin{aligned} \limsup \nolimits _{\,i\,\rightarrow \,\infty }V_i(t_{i0},x_{i0})\le V(t_0,x_0)\;\;\;{ for\,\,some\,\,sequence}\;\;\; (t_{i0},x_{i0})\rightarrow (t_0,x_0). \end{aligned}$$

Proof

Fix $(t_0,x_0)\in [0,T]\times {\mathbb R}^n$. Without loss of generality, we may assume $V(t_0,x_0)<+\infty$ since otherwise there is nothing to prove. Then, by Corollary 4.1, there exists the optimal arc $(\bar{x},\bar{a})(\cdot )$ of $V(t_0,x_0)$ defined on $[t_0,T]$. We extend $\bar{a}(\cdot )$ from $[t_0,T]$ to [0, T] by setting $\bar{a}(t)=0$ for $t\in [0,t_0]$. Next, because of the sublinear growth of f, we extend $\bar{x}(\cdot )$ from $[t_0,T]$ to [0, T] such that $(\bar{x},\bar{a})(\cdot )\in S _f(t_0,x_0)$. By e-$\lim _{i\rightarrow \infty }g_i=g$, there exists a sequence $z_{i0}\rightarrow \bar{x}(T)$ such that $g_i(z_{i0})\rightarrow g(\bar{x}(T))$. Now we choose $z_i(\cdot )$ defined on [0, T] such that $(z_i,\bar{a})(\cdot )\in S _{f_i}(T,z_{i0})$. Let $M>0$ be a constant such that $z_{i0},x_0,\bar{x}(T)\in {\mathbb B}_M$. Applying Gronwall’s Lemma to $(\bar{x},\bar{a})(\cdot )\in S _f(T,\bar{x}(T))$ and $(z_i,\bar{a})(\cdot )\in S _{f_i}(T,z_{i0})$, similarly as (14) and (15), we get $\Vert z_i\Vert \vee \Vert \bar{x}\Vert \le R$,

$$\begin{aligned} \Vert z_i-\bar{x}\Vert \;\le \; \Big (|z_{i0}-\bar{x}(T)|+\Vert f_i[z_i]-f[z_i]\Vert _{L^1}\Big )D. \end{aligned}$$

(20)

Similarly to (17) we show that $\lim \nolimits _{i\rightarrow \infty }\Vert f_i[z_i]-f[z_i]\Vert _{L^1}=0$. The latter and (20), together with $z_{i0}\rightarrow \bar{x}(T)$, imply that $\lim \nolimits _{i\rightarrow \infty }\Vert z_i-\bar{x}\Vert =0$. Hence we obtain $z_i(t_0)\rightarrow \bar{x}(t_0)=x_0$. Moreover, similarly to (18), we can also show that $\lim \nolimits _{i\rightarrow \infty }\Vert l_i[z_i]-l[\bar{x}]\Vert _{L^1}=0$. Note that

$$\begin{aligned} V(t_0,x_0)= & {} g(\bar{x}(T))+\int \limits _{\scriptscriptstyle t_0}^{\scriptscriptstyle T}l[\bar{x}](t)\,\mathrm{d}t\\ {}\ge & {} g(\bar{x}(T))+\int \limits _{\scriptscriptstyle t_{0}}^{\scriptscriptstyle T}l_{i}[z_i](t)\,\mathrm{d}t-\Vert l_i[z_i]-l[\bar{x}]\Vert _{L^1}\\\ge & {} g(\bar{x}(T))-g_i(z_{i0})+ V_i(t_{0},z_i(t_0))-\Vert l_i[z_i]-l[\bar{x}]\Vert _{L^1}. \end{aligned}$$

Passing to the limit in the above inequality, we get $\limsup \nolimits _{\,i\,\rightarrow \,\infty }V_i(t_0,z_i(t_0))\le V(t_0,x_0)$, where $(t_0,z_i(t_0))\rightarrow (t_0,x_0)$. $\square$

6.2 Lower Semicontinuity of Value Functions

We consider the set-valued maps:

$$\begin{aligned} Q(t,x)&:=\{(w,\eta )\in {\mathbb R}^n\times {\mathbb R}: (w,-\eta )\in {\mathrm {epi}}\,H^{*}(t,x,\cdot )\},\\ Q_i(t,x)&:=\{(w,\eta )\in {\mathbb R}^n\times {\mathbb R}: (w,-\eta )\in {\mathrm {epi}}\,H_i^{*}(t,x,\cdot )\}. \end{aligned}$$

Lemma 6.1

Let $H_i,H$, $i\in {\mathbb N}$ be as in Theorem 4.5 and $Q_i,Q$ be as above. Assume that $(t_{i0},x_{i0},u_{i0})\rightarrow (t_0,x_0,u_0)$ and $t_{i0},t_0\in [0,T[$. We consider $(x_i,u_i)\in \mathcal {A}([t_{i0},T],{\mathbb R}^{n}\times {\mathbb R})$ such that

$$\begin{aligned} (\dot{x}_i,\dot{u}_i)(t)\in Q_i(t,x_i(t))\;{ a.e.}\;t\in [t_{i0},T],\;\;(x_i,u_i)(t_{i0})=(x_{i0},u_{i0}). \end{aligned}$$

(21)

Assume that $u_i(T)\ge M$ for all $i\in {\mathbb N}$ and some constant M. Then there exist a function $(x,v)\in \mathcal {A}([t_{0},T],{\mathbb R}^{n}\times {\mathbb R})$ and a real number $v_0\le u_0$ such that

$$\begin{aligned} (\dot{x},\dot{v})(t)\in Q(t,x(t))\;{ a.e.}\;t\in [t_{0},T],\;\;(x,v)(t_{0})=(x_{0},v_{0}). \end{aligned}$$

(22)

Moreover, there exist a subsequence $(x_{i_k},u_{i_k})$ of a sequence $(x_{i},u_{i})$ such that

$$\begin{aligned} \lim \nolimits _{k\rightarrow \infty }x_{i_k}(T)=x(T)\;\quad { and}\quad \; \lim \nolimits _{k\rightarrow \infty }u_{i_k}(T)\le v(T). \end{aligned}$$

(23)

The proof of Lemma 6.1 is similar to the proof of Lower Closure theorem from the monograph of Cesari [22], so we omit it (see [20, Sect. 8] for more details).

Theorem 6.3

Let $H_i,H$, $i\in {\mathbb N}$ be as in Theorem 4.5. Assume that $g_i$ and g are proper, lower semicontinuous functions and e-$\lim _{i\rightarrow \infty }g_i=g$. Let $V_i$ and V be the value functions associated with $(H_i^{*},g_i)$ and $(H^{*},g)$, respectively. Then, for every $(t_0,x_0)\in [0,T]\times {\mathbb R}^n$, we have

$$\begin{aligned} \liminf \nolimits _{\,i\,\rightarrow \,\infty }V_i(t_{i0},x_{i0})\ge V(t_0,x_0)\;\;\;{ for\,\,every\,\,sequence}\;\;\; (t_{i0},x_{i0})\rightarrow (t_0,x_0). \end{aligned}$$

Proof

Fix $(t_{i0},x_{i0})\rightarrow (t_0,x_0)$. Let $\Delta :=\liminf \nolimits _{\,i\,\rightarrow \,\infty }V_i(t_{i0},x_{i0})$. We show that $V(t_0,x_0)\le \Delta$. Without loss of generality, we may assume $t_{i0}<T$ and $\Delta <+\infty$. By definition of $\Delta$, there exists a subsequence (we do not relabel) such that $V_i(t_{i0},x_{i0})\rightarrow \Delta$. Hence $V_i(t_{i0},x_{i0})<+\infty$ for all large $i\in {\mathbb N}$. By Corollary 4.1 there exist $x_i(\cdot )\in \mathcal {A}([t_{i0},T],{\mathbb R}^{n})$ such that $x_i(t_{i0})=x_{i0}$ and

$$\begin{aligned} V_i(t_{i0},x_{i0})\;=\;g_i(x_i(T))+\int _{t_{i0}}^TH_i^{*}(t,x_i(t),\dot{x}_i(t))\,\mathrm{d}t. \end{aligned}$$

(24)

Since $V_i(t_{i0},x_{i0})<+\infty$ for all large $i\in {\mathbb N}$, we have $H_i^{*}(t,x_i(t),\dot{x}_i(t))<+\infty$ for a.e. $t\in [t_{i0},T]$ and all large $i\in {\mathbb N}$. The latter, together with (C5), implies that $|\dot{x}_i(t)|\le c(t)(1+|x_i(t)|)$ for a.e. $t\in [t_{i0},T]$ and all large $i\in {\mathbb N}$. Thus, because of Gronwall’s Lemma, for all large $i\in {\mathbb N}$,

$$\begin{aligned} \Vert x_i(\cdot )\Vert \le \big (\sup \nolimits _{i\in {\mathbb N}}\,|x_{i0}|+\Vert c\Vert _{L^1}\big )\,\exp \big (\Vert c\Vert _{L^1}\big )=:R. \end{aligned}$$

Hence $|\dot{x}_i(t)|\le (1+R)\,c(t)$ for a.e. $t\in [t_{i0},T]$ and all large $i\in {\mathbb N}$. By our assumptions, we have $H_i^{*}(t,x_i(t),\dot{x}_i(t))\ge -\mu _R(t)$ for a.e. $t\in [t_{i0},T]$ and all large $i\in {\mathbb N}$. By e-$\lim _{i\rightarrow \infty }g_i=g$, there exists a constant M such that $g_i(x)\ge M$ for all $x\in {\mathbb B}_R$ and all large $i\in {\mathbb N}$. Therefore, by (24), for all large $i\in {\mathbb N}$, $V_i(t_{i0},x_{i0})\ge M-\Vert \mu _R\Vert _{L^1}>-\infty$. Hence $\Delta >-\infty$.

Case 1 Let $t_0<T$. We put $u_{i0}:=V_i(t_{i0},x_{i0})$ for all large $i\in {\mathbb N}$ and $u_0:=\Delta$. We define $u_i(\cdot )\in \mathcal {A}([t_{i0},T],{\mathbb R})$, for all large $i\in {\mathbb N}$, by

$$\begin{aligned} u_i(t)\;=\;g_i(x_i(T))+\int _{t}^TH_i^{*}(s,x_i(s),\dot{x}_i(s))\,ds. \end{aligned}$$

We observe that $-\dot{u}_i(t)=H_i^{*}(t,x_i(t),\dot{x}_i(t))$ for a.e. $t\in [t_{i0},T]$ and $u_i(t_{i0})=V_i(t_{i0},x_{i0})=u_{i0}$ for all large $i\in {\mathbb N}$. It means that the sequence $(x_i,u_i)(\cdot )$ satisfies (21) for all large $i\in {\mathbb N}$. Moreover, $u_i(T)=g_i(x_i(T))\ge M$ for all large $i\in {\mathbb N}$. Therefore, by Lemma 6.1, there exist $(x,v)(\cdot )\in \mathcal {A}([t_{0},T],{\mathbb R}^{n}\times {\mathbb R})$ and a real number $v_0\le u_0$ such that (22) holds. By (22), we deduce that

$$\begin{aligned} \Delta \,=\, u_0\ge & {} v_0=v(t_0)=v(T)+\int _{t_0}^T\!\!-\dot{v}(t)\,\mathrm{d}t\nonumber \\\ge & {} v(T)+\int _{t_0}^TH^{*}(t,x(t),\dot{x}(t))\,\mathrm{d}t\nonumber \\\ge & {} v(T)-g(x(T))+V(t_0,x_0). \end{aligned}$$

(25)

Moreover, in view of (23) and e-$\lim _{i\rightarrow \infty }g_i=g$, we deduce that

$$\begin{aligned} v(T)\ge \lim _{k\rightarrow \infty }u_{i_k}(T)=\lim _{k\rightarrow \infty }g_{i_k}(x_{i_k}(T))\ge g(x(T)). \end{aligned}$$

(26)

Combining inequalities (25) and (26), we obtain $\Delta \ge V(t_0,x_0)$.

Case 2 Let us consider $t_0=T$. We observe that

$$\begin{aligned} |x_i(t_{i0})-x_i(T)|\le \int _{t_{i0}}^T|\dot{x}_i(t)|\le (1+R)\int _{t_{i0}}^Tc(t)\,\mathrm{d}t\;\rightarrow \;0. \end{aligned}$$

The latter, together with $x_i(t_{i0})=x_{i0}\rightarrow x_0$, implies that $x_i(T)\rightarrow x_0$. Therefore, in view of (24) and e-$\lim _{i\rightarrow \infty }g_i=g$, we have

$$\begin{aligned} \Delta =\lim _{i\rightarrow \infty }V_i(t_{i0},x_{i0})\ge & {} \liminf _{i\rightarrow \infty } g_i(x_i(T))+\liminf _{i\rightarrow \infty }\int _{t_{i0}}^TH_i^{*}(t,x_i(t),\dot{x}_i(t))\,\mathrm{d}t\\\ge & {} \liminf _{i\rightarrow \infty } g_i(x_i(T))+\liminf _{i\rightarrow \infty }\int _{t_{i0}}^T\!\!-\mu _R(t)\,\mathrm{d}t\\\ge & {} g(x_0)=V(T,x_0)=V(t_0,x_0). \end{aligned}$$

This completes the proof of the theorem. $\square$

6.3 Remarks

In the proof of stability of value functions, we used the formula (8) on the value function as well as the formula (7). We do that, because formulas (7) and (8) have advantages and drawbacks.

The advantages of the formula (8) are regularities of functions f and l such that: a sublinear growth of the function f with respect to the state variable, a sublinear growth of the function l with respect to the control variable and local Lipschitz continuity with respect to the state variable for both functions f and l. These regularities of functions f and l together with the extra-property allow us to prove upper semicontinuity of value functions. On the other hand, the problems appear in the proof of lower semicontinuity of value functions. They can be overcome using convexity and coercivity of the function l with respect to the control variable, see [23, 24]. However, in our case the function l does not possess these properties and it is a drawback of the formula (8).

Lower semicontinuity of value functions is proven using the formula (7). It is possible due to convexity and coercivity of the conjugate $H^{*}(t,x,\cdot )$. These properties of the conjugate $H^{*}$ are advantages of the formula (7). The example of the Hamiltonian H in Sect. 3 shows that the conjugate $H^{*}$ is an extended-real-valued function and discontinuous on the effective domain ${\mathrm {dom}}\,H^{*}$. These properties of the conjugate $H^{*}$ are drawbacks of the formula (7).

6.4 Lipschitz Continuity of the Value Function

Assume that (H1)–(H4) and (HLC) hold with integrable functions $c(\cdot )$, $k_R(\cdot )$, $H(\cdot ,0,0)$. Let g be a locally Lipschitz function. We consider the representation $({\mathbb R}^{n+1}\!\!,f,l)$ of H defined as in Theorem 4.1. Let $M>0$ and

$$\begin{aligned} R:=\big (M+\Vert c\Vert _{L^1}\big )\exp \big (\Vert c\Vert _{L^1}\big ),\;\;\;\; C_M:=\big (D_R+\Vert \omega _{\scriptscriptstyle R}\Vert _{L^1}\big )\exp \big (\Vert \omega _{\scriptscriptstyle R}\Vert _{L^1}\big ), \end{aligned}$$

where $\omega _{\scriptscriptstyle R}[\cdot ]=2|H(\cdot ,0,0)|+(10\,(n+1)\,k_R(\cdot )+2c(\cdot ))(1+R)$ and $D_R$ denotes the Lipschitz constant of g on $B_R$. Let us consider the following function

$$\begin{aligned} \alpha _{M}(t):=(1+C_{M})\int _0^t\omega _{\scriptscriptstyle R}[s]\,ds\;\;\;\text {for all}\;\;\;t\in [0,T]. \end{aligned}$$

Proposition 6.1

Consider $({\mathbb R}^{n+1}\!\!,f,l)$ as above. Assume that g is a real-valued lower semicontinuous function. If V is the value function associated with $({\mathbb R}^{n+1}\!\!,f,l,g)$, then V is a real-valued function on $[0,T]\times {\mathbb R}^n$.

Proof

Fix $t_0\in [0,T]$ and $x_0\in {\mathbb R}^n$. We show that $-\infty< V(t_0,x_0)<+\infty$. Observe that the first inequality follows from (11). It remains to prove the second inequality. Let $\tilde{a}(\cdot )\equiv 0$. Then there exists $\tilde{x}(\cdot )\in \mathcal {A}([t_0,T],{\mathbb R}^n)$ such that $\dot{\tilde{x}}(t)=f(t,\tilde{x}(t),\tilde{a}(t))$ for a.e. $t\in [t_0, T]$ and $x(t_0)=x_0$. In view of Theorem 4.1 (A2) and (HLC), we get that

$$\begin{aligned} l(t,\tilde{x}(t),\tilde{a}(t))\le & {} 2|H(t,\tilde{x}(t),0)|+2c(t)(1+|\tilde{x}(t)|)+3|\tilde{a}(t)|\\\le & {} 2|H(t,0,0)|+2k_{\Vert \tilde{x}\Vert }(t)\,\Vert \tilde{x}\Vert +2c(t)(1+\Vert \tilde{x}\Vert )=:\mu (t) \end{aligned}$$

for a.e. $t\in [t_0, T]$. Thus, $V(t_0,x_0)\le g(\tilde{x}(T))+\Vert \mu \Vert _{L^1}<+\infty$. $\square$

Proof of Theorem 4.6

Fix $t_0,s_0\in [0,T]$ and $x_0,y_0\in {\mathbb B}_M$. Then, by Corollary 4.1 there exists the optimal arc $(\bar{x},\bar{a})(\cdot )$ of $V(t_0,x_0)$ defined on $[t_0,T]$. We extend $\bar{a}(\cdot )$ from $[t_0,T]$ to [0, T] by setting $\bar{a}(t)=0$ for $t\in [0,t_0]$. Next, because of the sublinear growth of f, we extend $\bar{x}(\cdot )$ from $[t_0,T]$ to [0, T] such that $(\bar{x},\bar{a})(\cdot )\in S _f(t_0,x_0)$. Now we choose $y(\cdot )$ defined on [0, T] such that $(y,\bar{a})(\cdot )\in S _f(s_0,y_0)$. By Gronwall’s Lemma, we get $\Vert \bar{x}\Vert \vee \Vert y\Vert \le R$,

$$\begin{aligned}&\Vert \bar{x}-y\Vert \le \Big (|x_0-y_0|+\int \limits _{\scriptscriptstyle t_0\wedge s_0}^{\scriptscriptstyle t_0\vee s_0}\omega _{\scriptscriptstyle R}[t]\,\mathrm{d}t\Big )\,\exp \Big (\int \limits _{\scriptscriptstyle 0}^{\scriptscriptstyle T}\omega _{\scriptscriptstyle R}[t]\,\mathrm{d}t\Big ),\end{aligned}$$

(27)

$$\begin{aligned}&\int \limits _{\scriptscriptstyle t_0\wedge s_0}^{\scriptscriptstyle T}\big |l[\bar{x}](t)-l[y](t)\big |\,\mathrm{d}t\;\le \Vert \bar{x}-y\Vert \int \limits _{\scriptscriptstyle 0}^{\scriptscriptstyle T}\omega _{\scriptscriptstyle R}[t]\,\mathrm{d}t, \end{aligned}$$

(28)

where $l[x](\cdot ):=l(\cdot ,x(\cdot ),\bar{a}(\cdot ))$. To prove theorem, we consider two cases:

Case 1 Let $t_0\le s_0$. By Theorem 4.1 (A2), we have $l[\bar{x}](t)\ge -\omega _{\scriptscriptstyle R}[t]$ for all $t\in [0,T]$. The latter, together with (27) and (28), implies that

$$\begin{aligned}&V(s_0,y_0)V(t_0,x_0)\le g(y(T))+\int \limits _{\scriptscriptstyle s_0}^{\scriptscriptstyle T}l[y](t)\,\mathrm{d}t-g(\bar{x}(T))-\int \limits _{\scriptscriptstyle t_0}^{\scriptscriptstyle T}l[\bar{x}](t)\,\mathrm{d}t\\&\quad \le |g(\bar{x}(T))-g(y(T))|+ \int \limits _{\scriptscriptstyle s_0}^{\scriptscriptstyle T}\big |l[\bar{x}](t)-l[y](t)\big |\,\mathrm{d}t-\int \limits _{\scriptscriptstyle t_0}^{\scriptscriptstyle s_0}l[\bar{x}](t)\,\mathrm{d}t\\&\quad \le \Vert \bar{x}-y\Vert \Big (D_R+\int \limits _{\scriptscriptstyle 0}^{\scriptscriptstyle T}\omega _{\scriptscriptstyle R}[t]\,\mathrm{d}t\Big )+ \int \limits _{\scriptscriptstyle t_0}^{\scriptscriptstyle s_{0}}\omega _{\scriptscriptstyle R}[t]\,\mathrm{d}t\\&\quad \le C_M|x_0-y_0|+(1+C_M)\int \limits _{\scriptscriptstyle t_{0}}^{\scriptscriptstyle s_0}\omega _{\scriptscriptstyle R}[t]\,\mathrm{d}t\\&\quad = C_M|x_0-y_0|+ |\alpha _M(t_0)-\alpha _M(s_0)|. \end{aligned}$$

Case 2 Let $s_0\le t_0$. Then $\bar{a}(t)=0$ for all $t\in [s_0,t_0]$. By Theorem 4.1 (A2), we have $l[\bar{x}](t)\le \omega _{\scriptscriptstyle R}[t]+3|\bar{a}(t)|$ for all $t\in [0,T]$. The latter, together with (27) and (28), implies that

$$\begin{aligned}&V(s_0,y_0)-V(t_0,x_0)\le g(y(T))+\int \limits _{\scriptscriptstyle s_0}^{\scriptscriptstyle T}l[y](t)\,\mathrm{d}t-g(\bar{x}(T))-\int \limits _{\scriptscriptstyle t_0}^{\scriptscriptstyle T}l[\bar{x}](t)\,\mathrm{d}t\\&\quad \le |g(\bar{x}(T))-g(y(T))|+\int \limits _{\scriptscriptstyle s_0}^{\scriptscriptstyle t_0}l[\bar{x}](t)\,t+ \int \limits _{\scriptscriptstyle t_0}^{\scriptscriptstyle T}\big |l[\bar{x}](t)-l[y](t)\big |\,\mathrm{d}t\\&\quad \le \Vert \bar{x}-y\Vert \,\Big (D_R+\int \limits _{\scriptscriptstyle 0}^{\scriptscriptstyle T}\omega _{\scriptscriptstyle R}[t]\,\mathrm{d}t\Big )+ \int \limits _{\scriptscriptstyle s_0}^{\scriptscriptstyle t_{0}}\omega _{\scriptscriptstyle R}[t]\,\mathrm{d}t+3\int \limits _{\scriptscriptstyle s_0}^{\scriptscriptstyle t_{0}}|\bar{a}(t)|\,\mathrm{d}t\\&\quad \le C_M|x_0-y_0|+(1+C_M)\int \limits _{\scriptscriptstyle s_{0}}^{\scriptscriptstyle t_0}\omega _{\scriptscriptstyle R}[t]\,\mathrm{d}t+3\int \limits _{\scriptscriptstyle s_0}^{\scriptscriptstyle t_{0}}|\bar{a}(t)|\,\mathrm{d}t\\&\quad = C_M|x_0-y_0|+ |\alpha _M(t_0)-\alpha _M(s_0)|. \end{aligned}$$

In view of Case 1 and Case 2, we conclude that the inequality (9) is true. If $c(\cdot )$, $k_R(\cdot )$, H are continuous, so is $\omega _{\scriptscriptstyle R}(\cdot )$. In this case, we show that V is Lipschitz continuous on $[0,T]\times {\mathbb B}_R$. Because of (9), it suffices to note that $|\alpha _M(t_0)-\alpha _M(s_0)|\le (1+C_M)\Vert \omega _R\Vert |t_0-s_0|$ for all $t_0,s_0\in [0,T]$, $x_0,y_0\in {\mathbb B}_M$. This completes the proof of the theorem. $\square$

7 Conclusions

In the case of representations with a compact control set, we knew what type of regularity we should expect, because this kind of representations had been considered by Frankowska–Sedrakyan and Rampazzo. Moreover, we found their broad applications in the monograph Bardi and Capuzzo-Dolcetta. However, in Theorem 4.1 the case of regularities of representations with the unbounded control set is much more complicated. Rampazzo-Sartori applied such representations in studies on regularity of value functions. However, they assumed coercivity of the function $l(t,x,\cdot )$. Unfortunately, the function $l(t,x,\cdot )$ from our faithful representation does not have this property. Therefore, their proofs cannot be applied in our case. This problem has been solved due to the extra-property and upper-boundedness of the function l. Investigating applications of representations with the unbounded control set leads us to the fundamental relation between variational and optimal control problems, see Theorem 4.4. The correlation between variational and optimal control problems has not been used earlier. For the first time, we have used this correlation in the proof of Theorem 4.5. Significant differences between representations with compact and unbounded control sets triggered us to write two distinct papers containing results related to them.