The goal of this article is to investigate new and simple convergence analysis of dynamic programming for the linear–quadratic regulator problem of discrete-time linear time-invariant systems. In particular, bounds on errors are given in terms of both matrix inequalities and matrix norm. Under a mild assumption on the initial parameter, we prove that the Q -value iteration exponentially converges to the optimal solution. Moreover, a global asymptotic convergence is also presented. These results are then extended to the policy iteration. We prove that in contrast to the Q -value iteration, the policy iteration always converges exponentially fast. An example is given to illustrate the results.

中文翻译：

---

离散时间无限视野LQR半定锥动态规划的收敛性


本文的目标是研究离散时间线性时不变系统的线性二次调节器问题的动态规划的新的简单收敛分析。特别是，误差范围是根据矩阵不等式和矩阵范数给出的。在初始参数的温和假设下，我们证明 Q 值迭代呈指数收敛于最优解。此外，还提出了全局渐近收敛。然后将这些结果扩展到策略迭代。我们证明，与 Q 值迭代相比，策略迭代总是以指数速度快速收敛。给出一个例子来说明结果。