Abstract
This paper expands the existing sufficiency results for the strong minimality of an extremal of the Bolza problem. We cover the case where the strict Legendre Clebsh conditions are not strictly verified. An efficient, easy to use, algorithm to prove minimality is provided. It can be used on solutions with bang-bang control and does not require any local controllability property. The interval where sufficiency is provided is maximized over a class of discontinuous Verification Functions determined solving a sequence of Riccati problems. This class of Verification Functions can be adapted to all kind of boundary conditions. The algorithms developed in this paper are applied to space trajectory extremals that exhibit bang-bang control.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
When \(dim(E)=d < n\), the existence of such Value Function requires a controllability property called Normality (see [12] Ch.5). When we have Normality such Value Function is both N.C. and S.C., our approach does not require the Normality because we look for subsolutions of the HJB.
This reference gives the S.C. at the control switching times in Hamiltonian form. The approach allows to reduce formally the problem to a finite dimensional minimization where the switching times are the free variables. Also in our method the S.C. are written at the switching times because the Riccati equation in this case is linear. However, a comparative study of the results has not been done.
\((x_t,u_t)\) the couple state and control denotes the functions or their values at time t, we use as well (x, u) to denote the functions or their generic value, so we may have some ambiguity that is solved looking at the operator applied to the couple.
The symbol \(| \ |\) is used to mean the Euclidean Norm.
A couple that verifies the N.C., may be normal or abnormal, we study the normal case.
We refer to the Clarke generalized gradient see [16], which is a non-empty closed convex set; we use the same symbol to mean the generalized gradient set and the partial derivative, the ambiguity being resolved by the logical statement.
When E is a moving target defined by \(\theta _i(x,t)=0, i=1,\ldots ,n-d\) we have that \(g(x,t)=l(x,t)+\sum _{i=1}^{n-d} \nu _{i} \theta _i(x,t) \). To deal with it, we augment the state as in [15] Ch.22.1, considering the time as a new state variable. The \(\theta _i(x,t)=0, i=1,\ldots ,n-d\) must have independent gradients in \(\mathbb {R}^{n+1}\).The N.C. of the Open Time case, result changed in the terminal condition of the Hamiltonian: \( H(x,p)|_{(x_f,p_f)} - \partial _t g(x,t)|_{(x_f,t_f)} =0\). The S.C. have to be applied to the augmented system. When the Hamiltonian is time dependent and the target is fixed, it is not necessary to use the augmented system, see the notes in Chapter 3.
From the Maximum Principle, \(H_P(\hat{z}_t,\hat{u}_{t_{c}+}) \ge H_P(\hat{z}_t,\hat{u}_{t_{c}-} )\) in a right interval of \(t_c\) and \(H_P(\hat{z}_{t_c},\hat{u}_{t_{c}+})= H_P(\hat{z}_{t_c},\hat{u}_{t_{c}-})\), then \(\partial _z H_P(\hat{z}_{t_c},\hat{u}_{t_{c}+}) \dot{\hat{z}}_{t_{c}+} \ge \partial _z H_P(\hat{z}_{t_c},\hat{u}_{t_{c}-}) \dot{\hat{z}}_{t_{c}+} \). From this follows that \(\frac{d}{dt}H_{k+1} (\hat{z}_t)|_{t_{c_{k}}+} \ge \frac{d}{dt}H_{k} (\hat{z}_t)|_{t_{c_{k}}+}\) because \(\partial _z H=\partial _z H_P(z,u)|_{u=F_b(z)}\) ([16] Th.2.1.)
This representation structurally satisfies the Maximum Principle and is a general presentation of the H0/I Hamiltonians close to cross points. It is easy to verify that \(H(z)=H_0(z)-k(S) S(z)\) does not agree with the Maximum Principle.
Ref. [4] shows that the strict Legendre–Clebsh conditions \(\partial _{uu} H_{P}(x,p,u) < 0 \), when the control set U is a compact convex polyhedron and the data \((f,\varLambda )\) are regular and autonomous, ensure that the Hamiltonian has Lipschitz first derivatives and a feedback which is Lipschitz in (x, p) and continuous along the optimal trajectory.
Any selection of \(\hat{u}_t \in F_b(\hat{x}_t,\hat{p}_t)\) is equivalent in the context of hypotheses H0/I.
We indicate with the small parameter \(\epsilon \) the tube where we construct the V.F.
The derivatives of this function that will be used later are: \( \partial _t \phi = \dot{\hat{p}}^T_t (x - \hat{x}_t)-\hat{p}^T_t \dot{\hat{x}}_t - (1/2) (x - \hat{x}_t)^T \dot{Q}_t (x - \hat{x}_t) + \dot{\hat{x}}^T_t Q_t (x - \hat{x}_t) \partial _x \phi = \hat{p}_t - Q_t (x - \hat{x}_t), \ \partial _{xt} \phi = \dot{\hat{p}}_t - \dot{Q}_t (x - \hat{x}_t) + Q_t \dot{\hat{x}}_t, \ \partial _{xx} \phi = - Q_t \)
The derivatives of G(x, t) are (the derivatives of H are calculated in \((x,p=\partial _x \phi (x,t),t)\)): \(\partial _x{G}=\partial _{xt}{\phi }+\partial _{xx}{\phi }\partial _{p}{H}+\partial _{x}{H} =\dot{\hat{p}}_t - \dot{Q}_t (x - \hat{x}_t) + Q_t \dot{\hat{x}}_t - Q_t \partial _{p}{H} + \partial _{x}{H} \partial _{xx}{G}= - \dot{Q}_t - Q_t \partial _{p x}{H} + Q^T_t \partial _{p p}{H} Q_t + \partial _{xx}{H} - \partial _{x p}{H} Q_t \)
In the Hypotheses H0/R with the compactness of U, \(G(\hat{x}_t,t)=sup_{u \in U} ( \partial _t \phi + {\partial _x \phi } ^{T} f(x,u)-\varLambda (x,u))|_{x=\hat{x}_t,t}=( \partial _t \phi + {\partial _x \phi } ^{T} f(x,u)-\varLambda (x,u))|_{x=\hat{x}_t,t,u=F_b(\hat{x}_t,\hat{p}_t)}\). So, by showing \( G(\hat{x}_t,t) \ge G(x,t), \ \forall x \in B_{\epsilon }(\hat{x}_t), \ \ t \in [t_0,t_f]\) we demonstrate that \(\phi (x,t)\) is a V.F. because it satisfies Ieq. (4).
\( A_{ij}(t) = \frac{\partial ^2{H}}{\partial {p_i}\partial {x_j}}|_{\hat{x}_t,\hat{p}_t} \ i=1,\ldots ,n,j=1,\ldots ,n , \, B_{ij}(t) = \frac{\partial ^2{H}}{\partial {p_i}\partial {p_j}}|_{\hat{x}_t,\hat{p}_t} \ i=1,\ldots ,n,j=1,\ldots ,n, \, C_{ij}(t) = \frac{\partial ^2{H}}{\partial {x_i}\partial {x_j}}|_{\hat{x}_t,\hat{p}_t} \ i=1,\ldots ,n,j=1,\ldots ,n \)
From H0/R follows that \(A,B,C \in C^0([t_0,t_f],\mathbb {R}^{(n,n)})\).
The S.C. of this problem are applicable also to time dependent Hamiltonians.
\(\dot{q}(t_{f})= (\partial _t{\phi } + H)|_{\hat{x}_{t_{f}},\hat{p}_{t_{f}},t_{f}} , \ddot{q}(t_{f})= (\partial _{tt}{\phi } + \partial _{tx}{\phi } \partial _{p} H +\partial _{t} H)|_{\hat{x}_{t_{f}},\hat{p}_{t_{f}},t_{f}}\), the last term applicable for time dependent Hamiltonian.
\( \partial _{xx} {[g(x) + \phi (x,t) - q(t)]}=\partial _{xx}g - Q_{\hat{t}_{f}}, \, \partial _{xt} {[g(x) + \phi (x,t) - q(t)]}=\dot{\hat{p}} + Q_{\hat{t}_{f}} \dot{\hat{x}}, \partial _{tt} {[g(x) + \phi (x,t) - q(t)]}= -(\dot{\hat{p}} + Q_{\hat{t}_{f}} \dot{\hat{x}})^T \dot{\hat{x}}-\partial _{t}H\). the last term applicable in case of time dependent Hamiltonians.
To clarify the notation used \(\partial _{\sigma ,\sigma }g=\varSigma ^T \partial _{x,x}g \varSigma \) or \(x_{\sigma }=\varSigma ^T x\), so \(\sigma ,\pi ,\mu \) are multi-indices that indicate the components of a matrix after a transformation by \(\varSigma , \varPi , M\); \(\nu \) is a unit vector, when used as subscript we mean the scalar resulting from the projection along its direction; \(0_{1,\pi }=0_{\nu ,\pi }=0_{1,dim(\pi )}\), \(I_{\sigma ,\sigma }\) is the identity matrix of dimension \(dim(\sigma )\).
When the Hamiltonian is time dependent the (S.C) are expressed changing in Ieq. (12) \(-{\dot{\hat{p}}}^{T}{\dot{\hat{x}}}\) with \(-{\dot{\hat{p}}}^{T}{\dot{\hat{x}}}- \partial _t H \), in Ieq. (13) \(-{\dot{\hat{p}}}_{\sigma }^{T}{\dot{\hat{x}}}_{\sigma }\) with \(-{\dot{\hat{p}}}_{\sigma }^{T}{\dot{\hat{x}}}_{\sigma } - \partial _t H \) and in Ieqs. (14) and (15) changing \(\dot{\hat{x}}_{\sigma }^T \partial _{\sigma ,\sigma }g \dot{\hat{x}}_{\sigma }+ {\dot{\hat{p}}}_{\sigma }^{T}{\dot{\hat{x}}}_{\sigma }\) with \(\dot{\hat{x}}_{\sigma }^T \partial _{\sigma ,\sigma }g \dot{\hat{x}}_{\sigma }+ {\dot{\hat{p}}}_{\sigma }^{T}{\dot{\hat{x}}}_{\sigma }-\partial _t H \),and in Ieq. (19) \( \zeta ^{T}_{\sigma }\partial _{\sigma ,\sigma }g \zeta _{\sigma } + \zeta ^{T}_{\sigma } \lambda _{\sigma } - \zeta ^{T}_{\pi } \lambda _{\pi }\) with \( \zeta ^{T}_{\sigma }\partial _{\sigma ,\sigma }g \zeta _{\sigma } + \zeta ^{T}_{\sigma } \lambda _{\sigma } - \zeta ^{T}_{\pi } \lambda _{\pi }-\partial _t H/ |\dot{\hat{x}}_{\pi }|^2 \).
Pre-multiplying each member of Ieq. (13) by R and post-multiplying by \(R^T\) we get:
$$\begin{aligned} R=\left( \begin{array}{cc} I_{\sigma ,\sigma } &{} 0_{\sigma ,1} \\ {\dot{\hat{x}}}^{T}_{\sigma } &{} -1 \end{array} \right) \rightarrow \left( \begin{array}{cc} \partial _{\sigma ,\sigma }g &{} \partial _{\sigma ,\sigma }g \dot{\hat{x}}_{\sigma } +{\dot{\hat{p}}}_\sigma \\ (\partial _{\sigma ,\sigma }g \dot{\hat{x}}_{\sigma } +{\dot{\hat{p}}}_\sigma )^T &{}\dot{\hat{x}}_{\sigma }^T \partial _{\sigma ,\sigma }g \dot{\hat{x}}_{\sigma }+ {{\dot{\hat{p}}}_{\sigma }^{T}{\dot{\hat{x}}}_{\sigma } } \end{array} \right) > \left( \begin{array}{cc} Q_{\sigma ,\sigma } &{} 0_{\sigma ,1} \\ 0_{1,\sigma } &{} 0 \end{array} \right) . \end{aligned}$$\(\nu \) is also used as subscript to mean the component of a matrix or vector along the \(\nu \) direction.
It follows:
$$\begin{aligned} \tau ^{T }\varGamma \tau = (u^T,w)\left( \begin{array}{cc} \partial _{\sigma ,\sigma }g &{} -\partial _{\sigma ,\sigma }g \zeta _{\sigma }- \lambda _{\sigma } \\ (-\partial _{\sigma ,\sigma }g \zeta _{\sigma }- \lambda _{\sigma })^T &{} \zeta ^{T}_{\sigma }\partial _{\sigma ,\sigma }g \zeta _{\sigma } + \zeta ^{T}_{\sigma } \lambda _{\sigma } - \zeta ^{T}_{\pi } \lambda _{\pi } \end{array} \right) \left( \begin{array}{cc} u \\ w \end{array} \right) \end{aligned}$$(18)o() means that for all successions with initial conditions \((\delta x^k_{0}, \delta p^k_{0} )\) converging to zero with \(k \rightarrow \infty \), \(|\delta x^k_t- (x^k_t-\hat{x}_t)|/(|\delta x^k_{0}|+|\delta p^k_{0}|) \rightarrow 0\) for all t in a compact interval.
\(\partial _{\alpha ,\beta }\hat{H}_t\) are the second derivatives of the Hamiltonian calculated on the candidate extremal arc, they define \(A_t,B_t,C_t\) as in Eq. (9)
The results from [19] have been established for continuous matrices \(A_t,B_t,C_t\) we will extend later these results for Irregular Hamiltonians.
The Hamiltonian is convex -not necessarily strictly- in the co-state, therefore \( \partial _{pp}H=B(x,p) \ge 0, \ (x,p) \in N_{\epsilon } , \, B_t \ge 0, \ t \in [t_0,t_f]\).
This formula covers all possible transitions considering that \(\dot{S}\) can be positive or negative, the formulas that follows are applicable in both cases.
When the Hamiltonian is time dependent the only difference in the S.C. consists in using for the derivative of S(x, t): \(\dot{S}|_{t_c} =(\partial _{x}S^{T}\partial _{p} H_0 -\partial _{p}S^{T} \partial _{x} H_0 + \partial _{t}S)|_{\hat{x}_{t_c},\hat{p}_{t_c}}\).
\(\varDelta \) gives the jump of the variable in the time positive direction, i.e.,\( \delta x|_{t}=\delta x|_{t+}-\delta x|_{t-}\)
\(\phi _S \in C^{1+}\) means that \(\partial _{(x,t)}\phi _S \in Lip\)
Where k is the index of the crossing point.
Let \(f \in C^k(\mathbb {R}^n)\) be the second member of an ODE \(\dot{y}=f(y), y(0)=y_0\). There is a neighborhood \(\varOmega \times I\) of \((y_0,0)\) where the solution \(y(t,y_0),\dot{y}(t,y_0)\) is in \(C^k(\varOmega \times I).\)
Let \(y \in C^k(\varOmega )\), and \(x \in \varOmega \), then \(y(x)=P_{k-1}(x-x_0)+R_k(x)\), where \(P_{k}(x-x_0)\) is the Taylor multivariate polynomial in \(x-x_0\) of order k and the residual \(R_k(x)=\sum _{|\alpha |=k} r_{\alpha }(x-x_0)^{k}, r_{\alpha }\in C^k(\varOmega )\) and \(|R_k(x)| \le C |x-x_0|^k\) for some C. When the derivatives of y(x) are available it can be written as \(r_{\alpha }(x)= D^{\alpha }y(x_0 + c(x-x_0))\), \(c \in (0,1)\).
We use in this passage the implicit function theorem that provides the \(C^k\) local regularity of the explicit function if the implicit is locally \(C^k\) and the Jacobian is not singular.
The modulus in \(|\dot{S}_0|\) takes into account that \(\partial _p{S}=sign(\dot{S}_0)(\dot{\hat{x}}_{0+}-\dot{\hat{x}}_{0-})\).
We introduce the apices \(+,-\) to the second-order residuals of the two fields.
\((\xi _{+} \ne \xi _{-} ,t)\) must be in the interior of \(D_{+}\) and \(D_{-}\) otherwise one of the Fields would not be individually injective.
For the Implicit function theorem.
\(C=sup_{x \in B_{\epsilon }(\hat{x}_{\delta })} |R_3(x-\hat{x}_{\delta },\delta )|/(\epsilon |x-\hat{x}_{\delta }|^2)\).
This can be seen from the matrix inverse calculated at Eq. (24).
See decomposition (20)
As a distribution \(\partial _{zz} H^{\delta } {\mathop {\rightarrow }\limits ^{\delta \rightarrow 0}} \partial _{zz} H_0 +\frac{\delta (t)}{|\dot{S}|}\partial _z S \partial ^{T}_{z} S + k(S)\partial _{zz} S \), however in the Jacobi Field it appears as a multiplicative coefficient. Here we just need to show that a regularization exists that converges to the Filippov solution of the Jacobi Field of H(z).
The lower degree polynomial solution of \(k_{\delta }(S)\) in \( -\delta \le S \le 0 \) is: \( k_{\delta }(S)=1+10 (\frac{S}{\delta })^3 + 15 (\frac{S}{\delta })^4 + 6 (\frac{S}{\delta })^5 \).
Proposition 4.2 is applicable for any \(\delta >0 \) because \(A^{\delta }_t,B^{\delta }_t,C^{\delta }_t\) are continuous
We mean that the restriction of the function to the segments \([-b,0 [\) and ]0, b] is \(C^1\).
Both matrices are bounded from above in the same interval by the solution of the Riccati equation having \(B=0\), because \(B \ge 0\).
\(H = \frac{\bar{H}^2}{2} + \bar{H} \), thus if \(\phi (x,t)\) verifies \(\partial _t \phi + H(x,\partial _x \phi ) = \partial _t \phi + \frac{\bar{H}^2(x,\partial _x \phi )}{2} + \bar{H}(x,\partial _x \phi ) \le 0\), it obviously verifies also: \(\partial _t \phi + \bar{H}(x,\partial _x \phi ) \le 0\). For what concerns the B.C. it is easy to check \(\partial _x H = \partial _x \bar{H}\) and \(\partial _p H = \partial _p \bar{H}\) for Open Time problems where \(H(x,p)=0\) along any extremal. So, the Hamiltonian H(x, p) can be used to derive both N.C. and S.C. of the Hamiltonian \(\bar{H}(x,p)\). In the following we use H(x, p) to derive both.
\(w_{*}(x,p)\) is calculated by the following conditions:\(\partial _w H_{P}(x,p,w) =0, \ \partial _{ww} H_{P}(x,p,w) < 0 \) by which we derive: \( \partial _x w_{*} = - \partial _{wx} H_P/ \partial _{ww} H_P, \ \partial _p w_{*} = - \partial _{wp} H_P / \partial _{ww} H_P \). The following Equations follow easily from the rules of differential calculus:
\( \partial _x H = \partial _x H_P + \partial _w H_P \partial _x w_{*}= \partial _x H_P, \ \partial _p H = \partial _p H_P + \partial _w H_P \partial _p w_{*}= \partial _p H_P \partial _{xx} H = \partial _{xx} H_P - \partial _{xw} H_P \partial _{xw} H_P^{T}/ \partial _{ww} H_P \) The same rules apply for the derivative \(\partial _{pp} , \partial _{xp} \) and we use them to calculate \(A_t,B_t,C_t\) along the extremal.
References
Hull, D.G.: Sufficient conditions for a minimum of the free-final-time optimal control problem. JOTA 68(2), 275–287 (1991)
Hull, D.G.: Optimal Control Theory for Applications. Springer, New York (2003)
Wood, L.J., Bryson, A.E.: Second order optimality conditions for variable end time optimal control problems. AIAA J. 11(9), 1241–1246 (1973)
Zeidan, V.: Extended Jacobi sufficiency criterion for optimal control. SIAM J. Control Optim. 22(2), 294–301 (1984)
Zeidan, V.: Sufficiency conditions with minimal regularity assumptions. Appl. Math. Optim. 20, 19–31 (1989)
Zeidan, V.: Sufficient conditions for the generalized problem of Bolza. Trans. Am. Math. Soc. 275(2), 561–586 (1983)
Zeidan, V., Chamberland, M.: Second order necessity and sufficiency theory for the free final time problem. In: Proceedings of the 31st IEEE Conference on Decision and Control, Tucson AZ (1992)
Maurer, H., Oberle, H.J.: Second order sufficient conditions for optimal control problems with free final time the riccati approach. SIAM J. Contin. Opt. 41(2), 380–403 (2002)
Osmolovskji, N.P., Maurer, H.: Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control. SIAM, Philipedia (2012)
Osmolovskji, N.P., Lempio, F.: Jacobi conditions and the Riccati equation for a broken extremal. J. Math. Sci. 100(5), 2572–2592 (2000)
Noble, J., Schattler, H.: Sufficient conditions for relative minima of broken extremals in optimal control theory. J. Math. Anal. Appl. 269, 98–128 (2002)
Schattler, H., Ledzewicz, U.: Geometric optimal control: theory, methods and examples. In: Interdisciplinary Applied Mathematics, vol. 38, Springer (2012)
Agrachev, A.A., Stefani, G., Zezza, P.: Strong optimality for a bang-bang trajectory. SIAM J. Control Optim. 41(4), 991–1014 (2002)
Vinter, R.: Optimal Control, Modern Birkhauser Classics. Springer, Berlin (2000)
Clarke, F.H.: Functional Analysis Calculus of Variation and Optimal Control. Springer, London (2013)
Clarke, F.H.: Generalized gradients and applications. In: Transactions of the America Mathematical Society, vol. 2015 (1975)
Luenberger, D.G.: Linear and Nonlinear Programming. Addison-Wesley, Boston (1984)
Filippov, A.F.: Differential Equations with Discontinuous Right Hand Sides. Springer, Berlin (1988)
Reid, W.T.: Riccati Differential Equations. Academic Press, New York (1972)
Clarke, F.H., Zeidan, V.: Sufficiency and the Jacobi conditions in the calculus of variations. Can. J. Math. XXXVIII(5), 1199–1209 (1986)
Milyutin, A.A., Osmolovskji, N.P.: Calculus of Variations and Optimal Control, American Mathematical Society, Translations of Mathematical Monographs, vol. 180 (1998)
Mazzini, L.: Time open orbital transfer in a transformed Hamiltonian setting. J. Guidance Control Dyn. 36(5), 1364–1374 (2013)
Mazzini, L.: Finite thrust orbital transfers. Acta Astron. 100, 107–128 (2014)
Mazzini, L.: Flexible Spacecraft Dynamics. Control and Guidance. Springer, Berlin (2016)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Mazzini, L. Extended Sufficient Conditions for Strong Minimality in the Bolza Problem: Applications to Space Trajectory Optimization. J Optim Theory Appl 191, 486–516 (2021). https://doi.org/10.1007/s10957-020-01798-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-020-01798-4