A stochastic resource allocation model, based on the principles of Markov decision processes ( MDPs ) , is proposed in this paper. In particular, a general-purpose framework is developed, which takes into account resource requests for both instant and future needs. The considered framework can handle two types of reservations ( i.e., specified and unspecified time interval reservation requests ) , and implement an overbooking business strategy to further increase business revenues. The resulting dynamic pricing problems can be regarded as sequential decision-making problems under uncertainty, which is solved by means of stochastic dynamic programming ( DP ) based algorithms. In this regard, Bellman’s backward principle of optimality is exploited in order to provide all the implementation mechanisms for the proposed reservation pricing algorithm. The curse of dimensionality, as the inevitable issue of the DP both for instant resource requests and future resource reservations, occurs. In particular, an approximate dynamic programming ( ADP ) technique based on linear function approximations is applied to solve such scalability issues. Several examples are provided to show the effectiveness of the proposed approach.

中文翻译：

---

随机资源分配问题的近似动态规划


本文提出了一种基于马尔可夫决策过程(MDP)原理的随机资源分配模型。特别是，开发了一个通用框架，该框架考虑了即时和未来需求的资源请求。所考虑的框架可以处理两种类型的预订（即指定和未指定时间间隔的预订请求），并实施超额预订业务策略以进一步增加业务收入。由此产生的动态定价问题可以看作是不确定性下的顺序决策问题，可以通过基于随机动态规划（DP）的算法来解决。在这方面，贝尔曼的后向最优性原理被利用，以便为所提出的预订定价算法提供所有实现机制。维数灾难是DP对于即时资源请求和未来资源预留不可避免的问题。特别地，应用基于线性函数近似的近似动态规划（ADP）技术来解决此类可扩展性问题。提供了几个例子来显示所提出方法的有效性。