Poisson’s equation has a lot of applications in various areas, such as Markov decision theory, perturbation theory, central limit theorems (CLTs), etc. Usually it is hard to derive the explicit expression of the solution of Poisson’s equation for a Markov chain on an infinitely many state space. Here we will present a computational framework for the solution for both discrete-time Markov chains (DTMCs) and continuous-time Markov chains (CTMCs), by developing the technique of augmented truncation approximations. The censored Markov chain and the linear augmentation to some columns are shown to be effective truncation approximation schemes. Moreover, the convergence to the variance constant in CLTs are also considered. Finally the results obtained are applied to discrete-time single-birth processes and continuous-time single-death processes.

泊松方程在各个领域有很多应用，例如马尔科夫决策理论、微扰理论、中心极限定理 (CLT) 等。 通常很难推导出马尔科夫链上泊松方程解的显式表达式。无限多个状态空间。在这里，我们将通过开发增强截断近似技术，为离散时间马尔可夫链 (DTMC) 和连续时间马尔可夫链 (CTMC) 提供一个计算框架。删失马尔可夫链和对某些列的线性增加被证明是有效的截断近似方案。此外，还考虑了对 CLT 中方差常数的收敛。最后将所得结果应用于离散时间的单生过程和连续时间的单生过程。