Skip to main content
Log in

Optimal Control of a Discrete-Time Stochastic System with a Probabilistic Criterion and a Non-fixed Terminal Time

  • topical issue
  • Published:
Automation and Remote Control Aims and scope Submit manuscript

Abstract

This paper considers an optimal control problem for a discrete-time stochastic system with the probability of first reaching the boundaries of a given domain as the optimality criterion. Dynamic programming-based sufficient conditions of optimality are formulated and proved. The isobells of levels 1 and 0 of the Bellman function are used for obtaining two-sided estimates of the right-hand side of the dynamic programming equation, two-sided estimates of the Bellman function, and two-sided estimates of the optimal-value function of the probabilistic criterion. A suboptimal control design method is proposed. The conditions of equivalence to an optimal control problem with a probabilistic terminal criterion are established. An illustrative example is given.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Afanas’ev, V.N., Kolmanosvkii, V.B., and Nosov, V.R.Matematicheskaya teoriya konstruirovaniya sistem upravleniya, Moscow: Vysshaya Shkola, 2003, 3rd ed. Translated under the title Mathematical Theory of Control Systems Design, Kluwer, 1996, 1st ed.

  2. Smirnov, I. P. On a Time-Optimal Control Problem for a Stochastic Controlled System. Differ. Uravn. 22(no. 2), 247–254 (1986).

    Google Scholar 

  3. Bertsekas, D. P. & Shreve, S. E. Stochastic Optimal Control: The Discrete-Time Case. (Academic, New York, 1978).

    MATH  Google Scholar 

  4. Zhou, J. Infinite Horizon Optimal Control Problem for Stochastic Evolution Equations in Hilbert Spaces. J. Dyn. Control Syst. 22(no. 3), 531–554 (2016).

    Article  MathSciNet  Google Scholar 

  5. Agram, N., Haadem, S., Øksendal, B. & Proske, F. A Maximum Principle for Infinite Horizon Delay Equations. SIAM J. Math. Anal. 45(no. 4), 2499–2522 (2013).

    Article  MathSciNet  Google Scholar 

  6. Agram, N. & Øksendal, B. Infinite Horizon Optimal Control of Forward-Backward Stochastic Differential Equations with Delay. J. Comput. Appl. Math. 259(part B), 336–349 (2014).

    Article  MathSciNet  Google Scholar 

  7. Chernous’ko, F. L. & Kolmanovskii, V. B. Optimalanoe upravlenie pri sluchainykh vozmushcheniyakh (Optimal Control under Random Perturbations). (Nauka, Moscow, 1978).

    Google Scholar 

  8. Smirnov, I. P. Control of the Probability of Reaching a Given Domain by a System. Differ. Uravn. 26(no. 10), 1753–1758 (1990).

    MATH  Google Scholar 

  9. Azanov, V. M. Optimal Control for Linear Discrete Systems with Respect to Probabilistic Criteria. Autom. Remote Control 75(no. 10), 1743–1753 (2014).

    Article  MathSciNet  Google Scholar 

  10. Semakov, S. L. Vybrosy sluchainykh protsessov: prilozheniya v aviatsii. (Nauka, Moscow, 2005). Translated under the title Crossings Problems in Random Processes Theory and Their Applications in Aviation, Newcastle: Cambridge Scholars Publisher, 2019.

    Google Scholar 

  11. Khametov, V. M. & Shelemekh, E. A. Superhedging of American Options on an Incomplete Market with Discrete Time and Finite Horizon. Autom. Remote Control 76(no. 9), 1616–1634 (2015).

    Article  MathSciNet  Google Scholar 

  12. Malyshev, V. V. & Kibzun, A. I. Analiz i sintez vysokotochnogo upravleniya letatelanymi apparatami (Analysis and Design of High-Precision Control of Aircrafts). (Mashinostroenie, Moscow, 1987).

    Google Scholar 

  13. Azanov, V. M. & Kan, Yu. S. Bilateral Estimation of the Bellman Function in the Problems of Optimal Stochastic Control of Discrete Systems by the Probabilistic Performance Criterion. Autom. Remote Control 79(no. 2), 203–215 (2018).

    Article  MathSciNet  Google Scholar 

  14. Azanov, V. M. & Kan, Yu. S. Refined Estimation of the Bellman Function for Stochastic Optimal Control Problems with Probabilistic Performance Criterion, Autom. Remote Control 80(no. 4), 634–647 (2019).

    Article  MathSciNet  Google Scholar 

  15. Grigor’ev, P. V. & Kan, Yu. S. Optimal Control of the Investment Portfolio with Respect to the Quantile Criterion. Autom. Remote Control 65(no. 2), 319–336 (2004).

    Article  MathSciNet  Google Scholar 

  16. Bunto, T. V. & Kan, Yu. S. Quantile Criterion-based Control of the Securities Portfolio with a Nonzero Ruin Probability. Autom. Remote Control 74(no. 5), 811–828 (2013).

    Article  MathSciNet  Google Scholar 

  17. Kibzun, A. I. & Ignatov, A. N. The Two-Step Problem of Investment Portfolio Selection from Two Risk Assets via the Probability Criterion. Autom. Remote Control 76(no. 7), 1201–1220 (2015).

    Article  MathSciNet  Google Scholar 

  18. Azanov, V. M. & Kan, Yu. S. Design of Optimal Strategies in the Problems of Discrete System Control by the Probabilistic Criterion. Autom. Remote Control 78(no. 6), 1006–1027 (2017).

    Article  MathSciNet  Google Scholar 

  19. Kibzun, A. I. & Ignatov, A. N. Reduction of the Two-Step Problem of Stochastic Optimal Control with Bilinear Model to the Problem of Mixed Integer Linear Programming. Autom. Remote Control 77(no. 12), 2175–2192 (2016).

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to V. M. Azanov.

Appendix

Appendix

Proof of Theorem 1. Introduce a function \({\Phi }_{0}:{{\mathbb{R}}}^{n}\to {\mathbb{R}}\) such that \({\Phi }_{0}\left(x\right)={\Phi }_{1}\left(x\right)\). Augment the state vector of the system by adding a new variable \({y}_{k}={\rm{min}}_{i = \overline{0,k}}{\Phi }_{i}\left({x}_{i}\right)\). The augmented control system has the form

$$\left\{\begin{array}{l}{x}_{k+1}={f}_{k}\left({x}_{k},{\widetilde{u}}_{k},{\xi }_{k}\right)\\ {y}_{k+1}={\rm{min}}\left\{{y}_{k},{\Phi }_{k}\left({x}_{k}\right)\right\}\\ {x}_{0}=X\\ {y}_{0}={\Phi }_{0}\left(X\right),\end{array}\right.\qquad k=\overline{0,N},$$

where \({\widetilde{u}}_{k}={\widetilde{\gamma }}_{k}\left({x}_{k},{y}_{k}\right)\). Consider the right-hand side \(\widetilde{f}:{{\mathbb{R}}}^{n+1}\times {{\mathbb{R}}}^{m}\times {{\mathbb{R}}}^{s}\to {{\mathbb{R}}}^{n+1}\) of the augmented system:

$${\widetilde{f}}_{k}\left(x,y,u,\xi \right)={\left({f}_{k}\left(x,u,\xi \right),{\rm{min}}\left\{y,{\Phi }_{k}\left(x\right)\right\}\right)}^{\,\,\,{\mathtt{T}}}.$$

Then the equivalent optimal control problem can be written as

$${\bf{P}}\left({\rm{min}}\left\{{y}_{N+1},{\Phi }_{N+1}\left({x}_{N+1}\right)\right\}\leqslant \varphi \right)\to \mathop{\rm{sup}}\limits_{u\left(\cdot \right)\in \widetilde{{\mathcal{U}}}},\quad k=\overline{0,N},$$

where \(\widetilde{{\mathcal{U}}}={\widetilde{{\mathcal{U}}}}_{0}\times \ldots \times {\widetilde{{\mathcal{U}}}}_{N}\) and

$${\widetilde{{\mathcal{U}}}}_{k}=\left\{\widetilde{\gamma }:{{\mathbb{R}}}^{n+1}\to {{\mathbb{R}}}^{m}| \widetilde{\gamma }\,\text{is Borel measurable}\,,\,\forall x\in {{\mathbb{R}}}^{n+1}:\widetilde{\gamma }\left(x\right)\in {U}_{k}\right\}.$$

Here the equivalence is understood in the sense of the same criteria

$$P\left(u\left(\cdot \right)\right)={\bf{P}}\left(\mathop{\rm{min}}\limits_{k = \overline{0,N}}{\Phi }_{k+1}\left(x\right)\leqslant \varphi \right)={\bf{P}}\left(\mathop{\rm{min}}\limits\left\{{y}_{N+1},{\Phi }_{N+1}\left({x}_{N+1}\right)\right\}\leqslant \varphi \right).$$

The Bellman equation for the equivalent problem [12] has the form

$${\widetilde{\gamma }}_{k}^{* }\left(x,y\right)={\rm{arg}}\,\mathop{\rm{max}}\limits_{u\in {U}_{k}}{{\bf{M}}}_{{\xi }_{k}}\left[{\widetilde{{\rm{B}}}}_{k+1}\left({\widetilde{f}}_{k}\left(x,y,u,{\xi }_{k}\right)\right)\right],$$
(A.1)
$${\widetilde{{\rm{B}}}}_{k}\left(x,y\right)=\mathop{\rm{max}}\limits_{u\in {U}_{k}}{{\bf{M}}}_{{\xi }_{k}}\left[{\widetilde{{\rm{B}}}}_{k+1}\left({\widetilde{f}}_{k}\left(x,y,{u}_{k},{\xi }_{k}\right)\right)\right],$$
(A.2)
$${\widetilde{{\rm{B}}}}_{N+1}\left(x,y\right)={{\bf{I}}}_{\left\{{\rm{min}}\left\{y,{\Phi }_{N+1}\left(x\right)\right\}\leqslant \varphi \right\}}\left(x,y\right),\quad k=\overline{0,N}.$$
(A.3)

The following result was established in [12]. Assume that:

1) The functions \({\widetilde{f}}_{k}\) are continuous for all \(k=\overline{0,N}\).

2) The function \({\rm{min}}\left\{y,{\Phi }_{N+1}\left(x\right)\right\}\) is continuous and bounded below.

3) The random vectors Xξ0, …, ξN are independent.

4) The sets U0, …, UN are compact.

Then the optimal control exists in the class of measurable functions and is determined by solving problems (A.1)–(A.3).

At step k = N the Bellman equation takes the form

$$\begin{array}{rcl}{\widetilde{{\rm{B}}}}_{N}\left(x,y\right)&=&\mathop{\rm{max}}\limits_{u\in {U}_{N}}{{\bf{M}}}_{{\xi }_{N}}\left[{{\bf{I}}}_{\left\{{\rm{min}}\left\{y,{\Phi }_{N}\left(x\right),{\Phi }_{N+1}\left(f\left(x,u,{\xi }_{k}\right)\right)\right\}\leqslant \varphi \right\}}\left(x,y\right)\right]\\ &=&\mathop{\rm{max}}\limits_{u\in {U}_{N}}{{\bf{M}}}_{{\xi }_{N}}\left[{{\bf{I}}}_{\left\{{\rm{min}}\left\{y,{\Phi }_{N}\left(x\right)\right\}\leqslant \varphi \right\}}\left(x,y\right)+\left(1-{{\bf{I}}}_{\left\{{\rm{min}}\left\{y,{\Phi }_{N}\left(x\right)\right\}\leqslant \varphi \right\}}\left(x,y\right)\right)\right.\\ &&\hspace{1em}\left.\times {{\bf{I}}}_{\left\{{\Phi }_{N+1}\left(f\left(x,u,{\xi }_{k}\right)\right)\leqslant \varphi \right\}}\left(x\right)\right]\\ &=&\mathop{\rm{max}}\limits_{u\in {U}_{N}}{{\bf{M}}}_{{\xi }_{N}}\left[{{\bf{I}}}_{\left\{{\rm{min}}\left\{y,{\Phi }_{N}\left(x\right)\right\}\leqslant \varphi \right\}}\left(x,y\right)+\left(1-{{\bf{I}}}_{\left\{{\rm{min}}\left\{y,{\Phi }_{N}\left(x\right)\right\}\leqslant \varphi \right\}}\left(x,y\right)\right)\right.\\ &&\hspace{1em}\left.\times {{\rm{B}}}_{N+1}\left({f}_{N}\left(x,u,{\xi }_{N}\right)\right)\right].\end{array}$$
(A.4)

Hence, for any \(k=\overline{0,N}\), Eq. (A.2) can be written as

$$\begin{array}{l}{{\rm{B}}_k}\left( {x,y} \right) = \mathop {{\rm{max}}}\limits_{u \in {U_k}} {{\bf{M}}_{{\xi _k}}}[{{\bf{I}}_{{\rm{\{ min}}\left\{ {{\rm{y}},{\Phi _{\rm{k}}}\left( {\rm{x}} \right)} \right\}\varphi \} }}\left( {{\rm{x}},{\rm{y}}} \right)\\\quad\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\left. {{\rm{ + }}\left( {{\rm{1 - }}{{\bf{I}}_{\left\{ {{\rm{min}}\left\{ {{\rm{y}},{\Phi _{\rm{k}}}\left( {\rm{x}} \right)} \right\}\varphi } \right\}}}\left( {{\rm{x}},{\rm{y}}} \right)} \right){{\rm{B}}_{{\rm{k + 1}}}}\left( {{{\rm{f}}_{\rm{k}}}\left( {{\rm{x}},{\rm{u}},{\xi _{\rm{k}}}} \right)} \right)} \right].\end{array}$$
(A.5)

Therefore, the Bellman function in the equivalent problem admits of the representation

$${\widetilde{{\rm{B}}}}_{k}(x,y)=\left\{\begin{array}{ll}1,&y\leqslant \varphi \\ \mathop{\rm{max}}\limits_{u\in {U}_{k}}{{\bf{M}}}_{{\xi }_{k}}\left\{{{\bf{I}}}_{{{\mathcal{F}}}_{k}}(x)+(1-{{\bf{I}}}_{{{\mathcal{F}}}_{k}}(x)){{\rm{B}}}_{k+1}({f}_{k}(x,u,{\xi }_{k}))\right\},&y>\varphi .\end{array}\right.$$
(A.6)

As a result, the optimal control is given by

$${\widetilde{\gamma }}_{k}^{* }(x,y)=\left\{\begin{array}{ll}\,\text{any element from}\,\,{U}_{k},&y\leqslant \varphi \\ {\rm{arg}}\,\mathop{\rm{max}}\limits_{u\in {U}_{k}}{{\bf{M}}}_{{\xi }_{k}}\left\{{{\bf{I}}}_{{{\mathcal{F}}}_{k}}(x)+(1-{{\bf{I}}}_{{{\mathcal{F}}}_{k}}(x)){{\rm{B}}}_{k+1}({f}_{k}(x,u,{\xi }_{k}))\right\},&y>\varphi .\end{array}\right.$$
(A.7)

Conditions 1–5 of Theorem 1 follow from the existence conditions of an optimal control in the problem with the terminal criterion. Note that items 2 and 3 are immediate from the continuity and boundedness (from below) of the function \({\rm{min}}\left\{y,{\Phi }_{N+1}\left(x\right)\right\}\).

The proof of Theorem 1 is complete.

Proof of Theorem 2. 1. Consider the Bellman equation at some step k. For the isobell of level 1,

$$\begin{array}{ccc}{{\mathcal{I}}}_{k}&=&\left\{x\in {{\mathbb{R}}}^{n}| \mathop{\rm{max}}\limits_{u\in {U}_{k}}{{\bf{M}}}_{{\xi }_{k}}\left[{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right)+\left(1-{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right)\right){{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)\right]=1\right\}\\ &=&{{\mathcal{F}}}_{k}\cup \left\{x\in {{\mathbb{R}}}^{n}| \mathop{\rm{max}}\limits_{u\in {U}_{k}}{{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)\right]=1\right\}.\end{array}$$

Using the total mathematical expectation formula, write

$$\begin{array}{cc}{{\mathcal{I}}}_{k}={{\mathcal{F}}}_{k}\cup \left\{x\in {{\mathbb{R}}}^{n}| \mathop{\rm{max}}\limits_{u\in {U}_{k}}\left\{{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right){{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)| {f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right]\right.\right.\\ \left.\left.+\left(1-{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)\right){{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)| {f}_{k}\left(x,u,{\xi }_{k}\right)\notin {{\mathcal{I}}}_{k+1}\right]\right\}=1\right\}.\end{array}$$

From the equalities

$$\begin{array}{l}{{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)| {f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right]=1,\\ {{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)| {f}_{k}\left(x,u,{\xi }_{k}\right)\notin {{\mathcal{I}}}_{k+1}\right]<1,\end{array}$$

it follows that

$$\begin{array}{ccc}{{\mathcal{I}}}_{k}={{\mathcal{F}}}_{k}\cup \left\{x\in {{\mathbb{R}}}^{n}| \mathop{\rm{max}}\limits_{u\in {U}_{k}}\left\{{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)+\left(1-{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)\right)\right.\right.\\ \left.\left.\times {{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)| {f}_{k}\left(x,u,{\xi }_{k}\right)\notin {{\mathcal{I}}}_{k+1}\right]\right\}=1\right\}\\ ={{\mathcal{F}}}_{k}\cup \left\{x\in {{\mathbb{R}}}^{n}| \mathop{\rm{max}}\limits_{u\in {U}_{k}}{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)=1\right\}.\end{array}$$

Item 1 of this theorem is established.

2. By analogy with item 1, the isobell of level 0 of the Bellman function can be written as

$$\begin{array}{ccc}{{\mathcal{O}}}_{k}&=&\left\{x\in {{\mathbb{R}}}^{n}| \mathop{\rm{max}}\limits_{u\in {U}_{k}}{{\bf{M}}}_{{\xi }_{k}}\left[{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right)+\left(1-{{\bf{I}}}_{{{\mathcal{F}}}_{k}}\left(x\right)\right){{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)\right]=0\right\}\\ &=&{\overline{{\mathcal{F}}}}_{k}\cap \left\{x\in {{\mathbb{R}}}^{n}| \mathop{\rm{max}}\limits_{u\in {U}_{k}}{{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)\right]=1\right\}.\end{array}$$

Using the total mathematical expectation formula, transform the right-hand side of this expression:

$$\begin{array}{rcl}{{\mathcal{O}}}_{k}&=&{\overline{{\mathcal{F}}}}_{k}\cap \left\{x\in {{\mathbb{R}}}^{n}| \mathop{\rm{max}}\limits_{u\in {U}_{k}}\left\{{{\bf{P}}}_{{\xi }_{k}}({f}_{k}(x,u,{\xi }_{k})\in {{\mathcal{I}}}_{k+1}){{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}({f}_{k}(x,u,{\xi }_{k}))| {f}_{k}(x,u,{\xi }_{k})\in {{\mathcal{I}}}_{k+1}\right]\right.\right.\\ &&+{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right){{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)| {f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right]\\ &&\left.\left.+{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right){{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)| {f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right]\right\}=0\right\}.\end{array}$$

From the equalities

$$\begin{array}{ccc}{{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)| {f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right]=1,\\ {{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)| {f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right]\in \left(0,1\right),\\ {{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)| {f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right]=0\end{array}$$

it follows that

$$\begin{array}{l}{{\mathcal{O}}}_{k}={\overline{{\mathcal{F}}}}_{k}\cap \left\{x\in {{\mathbb{R}}}^{n}| \mathop{\rm{max}}\limits_{u\in {U}_{k}}\left\{{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)\right.\right.\\ \left.\left.\quad\quad\quad\quad+{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right){{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)| {f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right]\right\}=0\right\}\\ \quad\quad={\overline{{\mathcal{F}}}}_{k}\cap \left\{x\in {{\mathbb{R}}}^{n}| \forall u\in {U}_{k}:{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)=0,\,{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right)=0\right\}\\ \quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad={\overline{{\mathcal{F}}}}_{k}\cap \left\{x\in {{\mathbb{R}}}^{n}| \forall u\in {U}_{k}:{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right)=1\right\}.\end{array}$$

Item 2 of this theorem is established.

3. Item 3 of this theorem follows from its item 1.

4. Item 4 of this theorem holds, since \({{\rm{B}}}_{k}\left(x\right)=0\) for \(x\in {{\mathcal{O}}}_{k}\).

5. Item 5 of this theorem follows its item 1.

6. For \(x\in {{\mathcal{B}}}_{k}\), the right-hand side of the dynamic programming equation can be represented as

$$\begin{array}{ccc}{{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)\right]={{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)\\ \times \left(1-{{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)| {f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right]\right)\\ +\left(1-{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right)\right){{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)| {f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{B}}}_{k+1}\right].\end{array}$$

Due to the two-sided inequality for a convex combination, this yields

$$\begin{array}{cc}{\rm{min}}\left\{{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right),\left(1-{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right)\right)\right\}{{\bf{M}}}_{{\xi }_{k}}\left[{{\rm{B}}}_{k+1}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)\right]\\ \leqslant {\rm{max}}\left\{{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right),\left(1-{{\bf{P}}}_{{\xi }_{k}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{O}}}_{k+1}\right)\right)\right\}.\end{array}$$

The relations \(1-{{\bf{P}}}_{{\xi }_{k}}({f}_{k}(x,u,{\xi }_{k})\in {{\mathcal{O}}}_{k+1})={{\bf{P}}}_{{\xi }_{k}}({f}_{k}(x,u,{\xi }_{k})\in {{\mathcal{I}}}_{k+1})+{{\bf{P}}}_{{\xi }_{k}}({f}_{k}(x,u,{\xi }_{k})\in {{\mathcal{B}}}_{k+1})\) and \({{\mathcal{F}}}_{k}\subseteq {{\mathcal{I}}}_{k}\) finally establish item 6 of this theorem.

7. Item 7 of this theorem is immediate from its item 6 by taking the supremum in all sides of inequality (8).

Proof of Theorem 3. 1. Introduce a system of hypotheses forming a complete group of incompatible events:

$$\left\{\mathop{\bigcup }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right\},\quad \left\{\mathop{\bigcap}\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right\}.$$

Then the total probability formula gives the chain of equalities

$$\begin{array}{ccc}P\left(\underline{u}\left(\cdot \right)\right)={\bf{P}}\left(\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right)\\ ={\bf{P}}\left(\mathop{\bigcup }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right|\mathop{\bigcup }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)\\ +\,\,{\bf{P}}\left(\mathop{\bigcap}\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right|\mathop{\bigcap}\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right).\end{array}$$
(A.8)

Analyze the second factor in the first term of the right-hand side of this expression. From the chain of equalities

$${\bf{P}}\left(\left.{\underline{x}}_{N+1}\in {{\mathcal{F}}}_{N+1}\right|{\underline{x}}_{N}\in {{\mathcal{I}}}_{N}\right)={\bf{P}}\left(\left.{\underline{x}}_{N+1}\in {{\mathcal{I}}}_{N+1}\right|{\underline{x}}_{N}\in {{\mathcal{I}}}_{N}\right)=1$$

it follows that

$${\bf{P}}\left(\left.\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right|\mathop{\bigcup }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)=1.$$

In view of this equality, the expression (A.8) takes the form

$$\begin{array}{cc}P\left(\underline{u}\left(\cdot \right)\right)={\bf{P}}\left(\mathop{\bigcup }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)\\ +\,\,{\bf{P}}\left(\mathop{\bigcap }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right|\mathop{\bigcap }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right).\end{array}$$
(A.9)

Now introduce another system of hypotheses forming a complete group of incompatible events:

$$\left\{\mathop{\bigcup }\limits_{k = 1}^{N-1}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right\},\quad \left\{\mathop{\bigcap }\limits_{k = 1}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right\}.$$

Apply the total probability formula for the first term in the right-hand side of (A.9):

$$\begin{array}{ccc}{\bf{P}}\left(\mathop{\bigcup }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)\\ ={\bf{P}}\left(\mathop{\bigcup }\limits_{k = 1}^{N-1}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcup }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right|\mathop{\bigcup }\limits_{k = 1}^{N-1}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)\\ +\,\,{\bf{P}}\left(\mathop{\bigcap }\limits_{k = 1}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcup }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right|\mathop{\bigcap }\limits_{k = 1}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right).\end{array}$$
(A.10)

Similar to (A.9), transform the right-hand side of (A.10) to obtain

$$\begin{array}{cc}{\bf{P}}\left(\mathop{\bigcup }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)={\bf{P}}\left(\mathop{\bigcup }\limits_{k = 1}^{N-1}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)\\ +\,\,{\bf{P}}\left(\mathop{\bigcap }\limits_{k = 1}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcup }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right|\mathop{\bigcap }\limits_{k = 1}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right).\end{array}$$
(A.11)

Substituting (A.10) into (A.9) yields

$$\begin{array}{ccc}P\left(\underline{u}\left(\cdot \right)\right)={\bf{P}}\left(\mathop{\bigcup }\limits_{k = 1}^{N-1}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right)\\ +\,\,{\bf{P}}\left(\mathop{\bigcap }\limits_{k = 1}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcup }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right|\mathop{\bigcap }\limits_{k = 1}^{N-1}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right)\\ +\,\,{\bf{P}}\left(\mathop{\bigcap }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right|\mathop{\bigcap }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right).\end{array}$$
(A.12)

Perform the analogous transformations for the first term in (A.12), introducing the systems of hypotheses

$$\left\{\mathop{\bigcup }\limits_{k = 1}^{l}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right\},\quad \left\{\mathop{\bigcap }\limits_{k = 1}^{l}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right\},\quad l=\overline{1,\ldots ,N-2},$$

to obtain the following expression for the value of the probabilistic criterion under the control \(\underline{u}\left(\cdot \right):\)

$$\begin{array}{ccc}P\left(\underline{u}\left(\cdot \right)\right)={\bf{P}}\left({\underline{x}}_{1}\in {{\mathcal{I}}}_{1}\right)\\ +\mathop{\sum }\limits_{l = 1}^{N-1}{\bf{P}}\left(\mathop{\bigcap }\limits_{k = 1}^{l}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcup }\limits_{k = 1}^{l+1}\left\{{\underline{x}}_{k}\in {{\mathcal{I}}}_{k}\right\}\right|\mathop{\bigcap }\limits_{k = 1}^{l}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right)\\ +\,\,{\bf{P}}\left(\mathop{\bigcap }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right){\bf{P}}\left(\left.\mathop{\bigcup }\limits_{k = 0}^{N}\left\{{\underline{x}}_{k+1}\in {{\mathcal{F}}}_{k+1}\right\}\right|\mathop{\bigcap }\limits_{k = 1}^{N}\left\{{\underline{x}}_{k}\notin {{\mathcal{I}}}_{k}\right\}\right).\end{array}$$
(A.13)

Note that the first term in (A.13) satisfies the chain of equalities

$${\bf{P}}\left({\underline{x}}_{1}\in {{\mathcal{I}}}_{1}\right)={\bf{P}}\left({f}_{0}\left(X,{\underline{u}}_{0},{\xi }_{0}\right)\in {{\mathcal{I}}}_{1}\right)={\underline{{\rm{B}}}}_{0}\left(X\right)=\underline{F}\left(\varphi ,N,X\right),$$

which implies the expression (14).

Item 1 of Theorem 3 is established.

2. For establishing item 2 of Theorem 3, it suffices to observe that for \({x}_{k}\in {{\mathcal{I}}}_{k}\), \({u}_{k}^{* }={\underline{u}}_{k}\) for all \(k=\overline{0,N}\). Following the same procedure as above, this equality can be used for deriving the expression (14) for the optimal-value function of the probabilistic criterion on the trajectories \({\left\{{x}_{k}^{* }\right\}}_{k = 1}^{N+1}\) of the closed loop system with the optimal control \({u}^{* }\left(\cdot \right)\).

Item 2 of Theorem 3 is established.

3. Item 3 of Theorem 3 directly follows from item 7 of Theorem 2 and item 1 of Theorem 3.

The proof of Theorem 3 is complete.

Proof of Lemma. Consider the dynamic programming relations (3)–(5) and (18)–(20) for problems (2) and (17), respectively. In (19), the equality

$${{\rm{B}}}_{k}^{\varphi }\left(x\right)=\mathop{\rm{max}}\limits_{u\in {U}_{k}}{\bf{M}}\left[{{\bf{I}}}_{{{\mathcal{I}}}_{k}^{\varphi }}\left(x\right)+\left(1-{{\bf{I}}}_{{{\mathcal{I}}}_{k}^{\varphi }}\left(x\right)\right){{\rm{B}}}_{k+1}^{\varphi }\left({f}_{k}\left(x,u,{\xi }_{k}\right)\right)\right],\quad k=\overline{0,N},$$

where \({{\mathcal{I}}}_{k}^{\varphi }\) is the isobell of level 1 of the Bellman function \({{\rm{B}}}_{k}^{\varphi }\left(x\right)\), holds for all \(x\in {{\mathbb{R}}}^{n}\). Hence, the equivalence conditions are true if the isobells of level 1 of the Bellman functions in problems (2) and (17) are the same, \({{\mathcal{I}}}_{k}={{\mathcal{I}}}_{k}^{\varphi }\). Due to the recurrent formula for \({{\mathcal{I}}}_{k}\) (see item 1 of Theorem 2), this is the case if and only if

$${{\mathcal{F}}}_{k}\subseteq \left\{x\in {{\mathbb{R}}}^{n}:\exists u\in {U}_{k}:{\bf{P}}\left({f}_{k}\left(x,u,{\xi }_{k}\right)\in {{\mathcal{I}}}_{k+1}\right)=1\right\},\quad k=\overline{0,N}.$$
(A.14)

The proof of Lemma is complete.

Proof of Proposition 1. In accordance with item 1 of Theorem 2, write the equations for the isobells of levels 1 and 0 of the Bellman function at step k = N:

$$\begin{array}{l}{{\mathcal{I}}}_{N}={\mathcal{F}}\cup \left\{x\in {\mathbb{R}}:\exists u\in U:{\bf{P}}\left(x\left(1+{u}^{1}b+\mathop{\sum }\limits_{i=2}^{m}{u}^{i}{\xi }_{N}^{i}\right)\geqslant -\varphi \right)=1\right\},\\ {{\mathcal{O}}}_{N}=\overline{{\mathcal{F}}}\cap \left\{x\in {\mathbb{R}}:\forall u\in U:{\bf{P}}\left(x\left(1+{u}^{1}b+\mathop{\sum }\limits_{i=2}^{m}{u}^{i}{\xi }_{N}^{i}\right)<-\varphi \right)=1\right\}.\end{array}$$

The solutions of these equations were obtained in [13]. Using those results and the notations for the boundaries of the sets \({{\mathcal{I}}}_{k}\) and \({{\mathcal{O}}}_{k}\) (see Section 6), write:

$$\begin{array}{l}{{\mathcal{I}}}_{N}=\left[\varphi ,+\infty \right)\cup \left[\varphi {\left(1+{\rm{max}}\left\{b,\mathop{\rm{max}}\limits_{j = \overline{2,m}}{\underline{b}}^{j}\right\}\right)}^{-1},+\infty \right)=\left[{\varphi }_{N}^{{\mathcal{I}}},+\infty \right),\\ {{\mathcal{O}}}_{N}=\left(-\infty ,\varphi \right]\cap \left(-\infty ,\varphi {\left(1+{\rm{max}}\left\{b,\mathop{\rm{max}}\limits_{j = \overline{2,m}}{\overline{b}}^{j}\right\}\right)}^{-1}\right]=\left(-\infty ,{\varphi }_{N}^{{\mathcal{O}}}\right].\end{array}$$

This implies \({{\mathcal{I}}}_{N}=\Delta {{\mathcal{I}}}_{N}\). Due to item 1 of Theorem 2, at step k = N − 1 the equations for the isobells take the form

$$\begin{array}{l}{{\mathcal{I}}}_{N-1}={\mathcal{F}}\cup \left\{x\in {\mathbb{R}}:\ \exists u\in U:\ {\bf{P}}\left(x\left(1+{u}^{1}b+\mathop{\sum }\limits_{i = 2}^{m}{u}^{i}{\xi }_{N-1}^{i}\right)\geqslant {\varphi }_{N}^{{\mathcal{I}}}\right)=1\right\},\\ {{\mathcal{O}}}_{N-1}=\overline{{\mathcal{F}}}\cap \left\{x\in {\mathbb{R}}:\forall u\in U:{\bf{P}}\left(x\left(1+{u}^{1}b+\mathop{\sum }\limits_{i = 2}^{m}{u}^{i}\ {\xi }_{N-1}^{i}\right)<{\varphi }_{N}^{{\mathcal{O}}}\right)=1\right\}.\end{array}$$

By analogy with step k = N, it follows that

$${{\mathcal{I}}}_{N-1}=\Delta {{\mathcal{I}}}_{N-1}=\left[{\varphi }_{N-1}^{{\mathcal{I}}},+\infty \right),\quad {{\mathcal{O}}}_{N-1}=\left(-\infty ,{\varphi }_{N-1}^{{\mathcal{O}}}\right].$$

Mathematical induction on k finally leads to the conclusion that, for all \(k=\overline{0,N}\),

$${{\mathcal{I}}}_{k}=\Delta {{\mathcal{I}}}_{k}=\left[{\varphi }_{k}^{{\mathcal{I}}},+\infty \right),\quad {{\mathcal{O}}}_{k}=\left(-\infty ,{\varphi }_{k}^{{\mathcal{O}}}\right].$$

The proof of Proposition 1 is complete.

Proof of Proposition 2. Consider the control (11), which takes the form (25); see Proposition 1. From item 3 of Theorem 3 it follows that

$$P\left(\underline{u}\left(\cdot \right)\right)={\bf{P}}\left(\mathop{\max }\limits_{k\in \left\{0,\ldots ,N\right\}}{\underline{x}}_{k+1}\geqslant -\varphi \right)\geqslant \underline{F}\left(\varphi ,N,X\right),$$
(A.15)

where the function \(\underline{F}\) is given by

$$\underline{F}\left(\varphi ,N,X\right)=\mathop{\rm{max}}\limits_{u\in U}{\bf{P}}\left(X\left(1+b{u}^{1}+\mathop{\sum }\limits_{j = 2}^{m}{u}^{j}{\xi }_{0}^{j-1}\right)\geqslant -\frac{\varphi }{{\left(1+{\rm{max}}\left\{b,\mathop{\rm{max}}\limits_{j = \overline{1,m-1}}{\underline{\varepsilon }}_{j}\right\}\right)}^{N}}\right)$$
(A.16)

due to

$${\varphi }_{1}^{{\mathcal{I}}}=-\varphi {\left(1+{\rm{max}}\left\{b,\mathop{\rm{max}}\limits_{j = \overline{1,m-1}}{\underline{\varepsilon }}_{j}\right\}\right)}^{-N}.$$

Clearly, the value \(\underline{N}\in {\mathbb{N}}\) can be determined as a root of the equation \(\underline{F}\left(\varphi ,N,X\right)=1\). But there exist infinitely many such roots, and the estimate \(\underline{N}\) will be therefore found in the form

$$\underline{N}={\rm{min}}\left\{N\in {\mathbb{N}}:\underline{F}\left(\varphi ,N,X\right)=1\right\}.$$
(A.17)

The expressions (A.15) and (A.17) imply

$${\bf{P}}\left(\mathop{\rm{max}}\limits_{k\in \left\{0,\ldots ,\underline{N}\right\}}{\underline{x}}_{k+1}\geqslant -\varphi \right)=1.$$

From the definition of the isobell of level 1 of the Bellman function and the definition of the function \(\underline{F}\) it follows that the equation \(\underline{F}\left(\varphi ,N,X\right)=1\) is equivalent to the inclusion \(X\in {{\mathcal{I}}}_{0}\), which is in turn equivalent to

$$X\geqslant -\frac{\varphi }{{\left(1+{\rm{max}}\left\{b,\mathop{\rm{max}}\limits_{j = \overline{1,m-1}}{\underline{\varepsilon }}_{j}\right\}\right)}^{N+1}}.$$

Taking the logarithm gives the inequality

$$N\geqslant \frac{\mathrm{ln}\,\left(-\varphi \right)-\mathrm{ln}\,\left(X\right)}{\mathrm{ln}\,\left(1+{\rm{max}}\left\{b,\mathop{\rm{max}}\limits_{j = \overline{1,m-1}}{\underline{\varepsilon }}_{j}\right\}\right)}-1.$$

In view of (A.17), this finally leads to (26).

The proof of Proposition 2 is complete.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Azanov, V. Optimal Control of a Discrete-Time Stochastic System with a Probabilistic Criterion and a Non-fixed Terminal Time. Autom Remote Control 81, 2143–2159 (2020). https://doi.org/10.1134/S0005117920120012

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0005117920120012

Keywords

Navigation