Markov decision models (MDM) used in practical applications are most often less complex than the underlying ‘true’ MDM. The reduction of model complexity is performed for several reasons. However, it is obviously of interest to know what kind of model reduction is reasonable (in regard to the optimal value) and what kind is not. In this article we propose a way how to address this question. We introduce a sort of derivative of the optimal value as a function of the transition probabilities, which can be used to measure the (first-order) sensitivity of the optimal value w.r.t. changes in the transition probabilities. ‘Differentiability’ is obtained for a fairly broad class of MDMs, and the ‘derivative’ is specified explicitly. Our theoretical findings are illustrated by means of optimization problems in inventory control and mathematical finance.

实际应用中使用的马尔可夫决策模型（MDM）通常比底层的“真正” MDM复杂。降低模型复杂度的原因有很多。但是，显然知道哪种模型简化是合理的（就最优值而言），哪种模型不可行。在本文中，我们提出了一种解决此问题的方法。我们介绍了一种最佳值的导数，它是过渡概率的函数，可用于测量过渡概率中最优值的（一阶）敏感性。对于相当广泛的一类MDM，获得了“可区分性”，并明确指定了“衍生物”。通过在库存控制和数学财务方面的优化问题来说明我们的理论发现。