Elsevier

Automatica

Volume 132, October 2021, 109806
Automatica

Optimal control for discrete-time NCSs with input delay and Markovian packet losses: Hold-input case

https://doi.org/10.1016/j.automatica.2021.109806Get rights and content

Abstract

This paper is concerned with the linear–quadratic optimal control problem for networked systems simultaneously with input delay and Markovian packet losses under hold-input compensation strategy, which is different from the literature. Necessary and sufficient conditions for the solvability of optimal control problem over a finite horizon are given by coupled difference Riccati-type equations. Moreover, the networked control system is mean-square stability if and only if coupled algebraic Riccati-type equations have a particular solution. Due to input delay and Markovian packet dropout, it leads to the failure of the separation principle, which is a fundamental obstacle. The key technique in this paper is to tackle forward and backward difference equations by decoupling.

Introduction

A network control system (NCS, for short), known as a communication and control system, is a fully distributed and networked real-time feedback control system. Since the concept of NCSs was proposed in the early 1990s, it has attracted attention. For example, Guo, Lu, and Shi (2014) discussed the networked control problems for discrete linear systems whose network mediums for the actuators are constrained. Using the Lyapunov–Krasovskii functional method, Yue, Han, and Lam (2005) considered the disturbance attenuation problem for NCSs. At the same time, it raised new challenges to traditional control theory and applications. In an NCS, multiple network nodes share a network channel. Due to the limited network bandwidth and irregular changes in data traffic in the network, data collisions and network congestion often occur when multiple nodes exchange data through the network. Therefore, packet losses and time delay will inevitably occur.

Generally, there are two kinds of packet losses. When the process of packet transmission is mutually independent, the packet loss is modeled as a Bernoulli process. Liang and Xu (2018) focused on the optimal control problems for NCSs, which are simultaneously controlled by the remote controller and the local controller. Lu, Zhong, and Qu (2019) considered the NCSs with Bernoulli packet losses using an improved switching hold compensation strategy in which the too old held signal is deleted. Based on the characteristic of independent and identical distribution, as said in Zhang, Li, Xu, and Fu (2015), it is easy to verify that NCSs with Bernoulli packet losses can be regarded as a special case of system with multiplicative white noise. However, if the current packet cases affect future packet cases, a Markov process can represent such effect rather than a Bernoulli process. In fact, it is more involved to be dealt with than Bernoulli packet losses due to the temporal correlation caused by Markovian characteristics. Wang, Wang, and Wang (2013) considered the H-controller design for NCSs with Markovian packet losses.  Xie and Xie (2009) presented the necessary and sufficient condition for the mean-square stability of sampled-data networked linear systems with Markovian packet losses.

For the packet loss case, some strategies are usually adopted to compensate in NCSs. The zero-input (i.e., zero value is directly adopted by the actuator input) and the hold-input (i.e., the latest available control signal stored in the actuator buffer is used) are two common compensation strategies. For zero-input case, Imer, Yuksel, and Basar (2006) discussed the optimal control problem for the linear system with packet losses under TCP and UDP protocols. Sufficient conditions for stability of network communication models with packet losses were studied by Montestruque and Antsaklis (2004). It must be pointed out that, as said in Guo et al. (2014) and Lu et al. (2019), the zero-input strategy is mainly for mathematical convenience as it gives simpler equations than hold-input strategy, rather than for performance considerations. However, in most practical applications, it is necessary to consider the performance. Moreover, using the latest control input stored in the actuator buffer provides better performance than using zero input, especially during transients, because the true current optimal control input is likely to be close to the previous value. Indeed, the hold-input strategy was studied in many previous works due to its universality in the practical field. For example, Hristu-Varsakelis (2008) analyzed the structure properties (e.g., observability and controllability) of the NCSs in which a zero-order hold is included.

Note that most of the aforementioned work only considered packet losses in NCSs (Liang and Xu, 2018, Lu et al., 2019, Wang et al., 2013, Xie and Xie, 2009, Zhan et al., 2015). Few works focus on the simultaneous occurrence of time-delay. Actually, when transmission delay occurs in NCSs with packet losses, the current controller will be designed with past information, leading to the controller’s adaptability problem. Therefore, some of the previous methods (e.g., the Smith predictor method Watanabe & Ito, 1981 which is only useful for the determinate or additional system) are no longer applicable. Although the time-delay system can be converted to a delay-free system by state augmentation (Delfour, 1984), it leads to the design of optimal controller feedback by a high-dimensional gain matrix and a large amount of calculation. What is more, it cannot reflect the influence of time delay on the optimal control problem in essence.

From the above-mentioned analysis, it is not easy to deal with the optimal control problem for NCSs with both input delay and Markovian packet losses. The fundamental difficulty of this problem can be attributed to the failure of the separation principle caused by simultaneous input delay and Markovian packet dropout. In this case, state prediction is required to design a controller, and the feedback gain matrix cannot be given by general Riccati equations. It is a fundamental challenge problem. Indeed, some studies have focused on the stabilization problem in the presence of both data packet dropout and delay due to its theoretical value and practical background. However, methods proposed in previous literature (e.g., predictive control method, Liu, 2010, and Lyapunov–Krasivskii functional approach, Yue et al., 2005, etc.) are mainly based on linear matrix inequalities. Only sufficient stabilizing conditions are available. The complete solution (i.e., sufficient and necessary conditions for the stabilization problem) are not derived at present. Hence, it is a fundamental challenge problem and has not been solved thoroughly.

In this paper, we consider the optimal control problem for NCSs with both input delay and Markovian packet losses under hold-input strategy. To reduce the computational complexity, the system under hold-input strategy is firstly converted to the linear system with Markovian jump (MJLS), which is another important topic (Costa et al., 2005, Li and Zhou, 2002, Li et al., 2003, Tian et al., 2020, Xue et al., 2020). For example, Li and Zhou (2002) and Li et al. (2003) considered the indefinite stochastic optimal control problems for the MJLS over a finite time horizon and an infinite time horizon, respectively. Also, Costa et al. (2005) studied discrete-time Markovian jump linear systems and their applications. In view of these, the key point in this paper is how to deal with the forward and backward stochastic difference equations (FBSDEs), which are derived by the stochastic maximum principle. Inspired by Zhang et al. (2015) and Zhang and Xu (2017) in which the FBSDEs have made substantial progress in optimal LQ control problem for linear systems, the main results in this paper are derived and can be summarized as follows. First, the necessary and sufficient conditions for the solvability of optimal control problem over a finite horizon are presented by the coupled difference Riccati-type equations (CDREs). Second, the existence of the solution to the coupled algebraic Riccati-type equations (CAREs) is proved. Moreover, the optimal controller and optimal cost functional over an infinite horizon are derived. Finally, the necessary and sufficient conditions for the stabilization of the NCSs are established using the CAREs.

The rest of this article is structured as follows. Section 2 gives the problem statement. Section 3 solves the optimal control problem over a finite horizon and the stabilization problems for the infinite horizon case. A numerical example is presented to verify the obtained results in Section 4. A summary is presented in Section 5. Proofs for some results can be found in Appendix A.

Notation

Rn is the n-dimensional Euclidean space and Rm×n the norm bounded linear space of all m×n matrices. Y is the transposition of Y and Y0(Y>0) means that YRn×n is symmetric positive semi-definite (positive definite). Let (Ω,F,Fk,P) be a complete probability space with the natural filtration {Fk}k0 generated by {θ0,,θk}. E[|Fk] means the conditional expectation with respect to Fk and F1 is understood as {,Ω}.

Section snippets

Problem statement and preliminaries

Consider the following discrete-time system: xk+1=Axk+Bukda,ukda=θkukdc+(1θk)ukd1a,k0,uia=uic,i=d,,1, where xkRn is the state. ukaRm denotes the control input to the actuator and ukcRm is the desired control input computed by the controller. The stochastic variable θk is the packet loss modeled as a two state Markov chain θk{0,1} with transition probability ξij=P(θk+1=j|θk=i)(i,j=0,1) between the controller and the actuator: Take uka=ukc, if the packet is correctly delivered;

Main results

For discussion, this section will follow two steps. The LQ optimal control problem over a finite horizon will be first considered. On this basis, Problem 1 will be resolved.

Example 1

Consider system (1) with A=1,B=15, and the initial values x0=0.001,u1c=0.2, let the transition probability ξ00=0.9,ξ11=0.7 and cost functional (6) with Q=1001,R=10. Therefore, Ā0=11501,Ā1=1000,B̄0=00,B̄1=151. In this case, a sample path of the Markov chain θk is shown in Fig. 1.

When d=1:

In view of (31)–(36), the following results can be obtained:

P0=3.7383251.49251.4969116.31,P1=3.804988.081888.081823233.77,Γ0=3006.02,Γ1=11877.83,M00=55.011383.48,M10=107.89461.16,M01=1834.41,M11=5058.20.

Conclusion

In this paper, the optimal LQ control problem for NCSs simultaneously with input delay and Markovian dropout is discussed. Compared with the literature results, we mainly consider the hold-input strategy, which is much more computationally complicated than zero-input strategy. Necessary and sufficient conditions for the solvability of optimal control problem over a finite horizon are presented by the CDREs. Moreover, the NCS is mean-square stability if and only if the CAREs have a particular

Hongdan Li received the B.S. and M.S. degree in mathematics from Qufu Normal University in 2013 and 2016, respectively, and the Ph.D. degree in the School of Control Science and Engineering, Shandong University in 2020. She is currently a Lecturer with College of Electrical Engineering and Automation, Shandong University of Science and Technology, Qingdao, Shandong. Her current research interests include optimal control, stochastic systems and time-delay systems and stabilization.

References (20)

There are more references available in the full text version of this article.

Cited by (10)

View all citing articles on Scopus

Hongdan Li received the B.S. and M.S. degree in mathematics from Qufu Normal University in 2013 and 2016, respectively, and the Ph.D. degree in the School of Control Science and Engineering, Shandong University in 2020. She is currently a Lecturer with College of Electrical Engineering and Automation, Shandong University of Science and Technology, Qingdao, Shandong. Her current research interests include optimal control, stochastic systems and time-delay systems and stabilization.

Xun Li received the B.S. degree from the Department of Mathematics, Shanghai University of Science and Technology, Shanghai, China, in 1992, the M.S. degree from the Department of Mathematics, Shanghai University, Shanghai, China, in 1995, and the Ph.D. degree from the Department of Systems Engineering and Engineering Management, Chinese University of Hong Kong, Hong Kong, China, in 2000. He was a Postdoctoral Research Fellow with the Chinese University of Hong Kong until 2001. From 2001 to 2003, he was a Postdoctoral Fellow in the Mathematical and Computational Finance Laboratory, University of Calgary, Calgary, AB, Canada. From 2003 to 2007, he was a Visiting Fellow in the Department of Mathematics, National University of Singapore. He joined the Department of Applied Mathematics, Hong Kong Polytechnic University as an Assistant Professor in 2007 and is currently Professor. He has published in journals such as the SIAM Journal on Control and Optimization, the Annals of Applied Probability, the IEEE TRANSACTIONS ON AUTOMATIC CONTROL, Automatica, Mathematical Finance, and Annals of Finance. His main research areas are applied probability and stochastic control with financial applications.

Huanshui Zhang graduated in mathematics from the Qufu Normal University in 1986 and received his M.Sc. and Ph.D. degrees in control theory from Heilongjiang University, China, and Northeastern University, China, in 1991 and 1997, respectively. He worked as a postdoctoral fellow at Nanyang Technological University from 1998 to 2001 and Research Fellow at Hong Kong Polytechnic University from 2001 to 2003. He is currently a Changjiang Professorship at Shandong University, China. He held Professor in Harbin Institute of Technology from 2003 to 2006. He also held visiting appointments as Research Scientist and Fellow with Nanyang Technological University, Curtin University of Technology and Hong Kong City University from 2003 to 2006. His interests include optimal estimation and control, time-delay systems, stochastic systems, signal processing and wireless sensor networked systems. He was an Associate Editor of the IEEE Transactions on Automatic Control, and the IEEE Transactions on Circuits and Systems I.

This work is supported by Research Grants Council of Hong Kong under grant 15213218 and 15215319, the National Natural Science Foundation of China under Grants U1701264 and the foundation for innovative research groups of national natural science foundation of China (61821004). The material in this paper was not presented at any conference. This paper was recommended for publication in revised form by Associate Editor Oswaldo Luiz V. Costa under the direction of Editor Sophie Tarbouriech.

View full text