We study Markov decision processes with Borel state spaces under quasi-hyperbolic discounting. This type of discounting nicely models human behaviour, which is time-inconsistent in the long run. The decision maker has preferences changing in time. Therefore, the standard approach based on the Bellman optimality principle fails. Within a dynamic game-theoretic framework, we prove the existence of randomised stationary Markov perfect equilibria for a large class of Markov decision processes with transitions having a density function. We also show that randomisation can be restricted to two actions in every state of the process. Moreover, we prove that under some conditions, this equilibrium can be replaced by a deterministic one. For models with countable state spaces, we establish the existence of deterministic Markov perfect equilibria. Many examples are given to illustrate our results, including a portfolio selection model with quasi-hyperbolic discounting.

我们研究准双曲折现下具有Borel状态空间的Markov决策过程。这种打折很好地模拟了人类的行为，从长远来看，这是时间不一致的。决策者的喜好会随时间变化。因此，基于Bellman最优性原理的标准方法失败了。在动态博弈论的框架内，我们证明了带有转移具有密度函数的一大类马尔可夫决策过程的随机平稳马尔可夫完美均衡的存在。我们还表明，在过程的每个状态下，随机化都可以限制为两个动作。此外，我们证明在某些条件下，这种平衡可以被确定性的平衡所替代。对于具有可数状态空间的模型，我们建立了确定性马尔可夫完美均衡的存在。