Consumption Markov Decision Processes (CMDPs) are probabilistic decision-making models of resource-constrained systems. In a CMDP, the controller possesses a certain amount of a critical resource, such as electric power. Each action of the controller can consume some amount of the resource. Resource replenishment is only possible in special reload states, in which the resource level can be reloaded up to the full capacity of the system. The task of the controller is to prevent resource exhaustion, i.e. ensure that the available amount of the resource stays non-negative, while ensuring an additional linear-time property. We study the complexity of strategy synthesis in consumption MDPs with almost-sure B\"uchi objectives. We show that the problem can be solved in polynomial time. We implement our algorithm and show that it can efficiently solve CMDPs modelling real-world scenarios.

中文翻译：

---

消费马尔科夫决策过程的定性控制器综合

消费马尔可夫决策过程 (CMDP) 是资源受限系统的概率决策模型。在 CMDP 中，控制器拥有一定数量的关键资源，例如电力。控制器的每个动作都会消耗一定数量的资源。资源补充只有在特殊的重新加载状态下才有可能，在这种状态下，资源水平可以重新加载到系统的全部容量。控制器的任务是防止资源耗尽，即确保可用资源量保持非负，同时确保额外的线性时间属性。我们研究了具有几乎确定的 B\"uchi 目标的消费 MDP 中策略合成的复杂性。我们表明该问题可以在多项式时间内解决。