In this paper, we propose an approximate dynamic programming (ADP) algorithm to solve a Markov decision process (MDP) formulation for the admission control of elective patients. To manage the elective patients from multiple specialties equitably and efficiently, we establish a waiting list and assign each patient a time-dependent dynamic priority score. Then, taking the random arrivals of patients into account, sequential decisions are made on a weekly basis. At the end of each week, we select the patients to be treated in the following week from the waiting list. By minimizing the cost function of the MDP over an infinite horizon, we seek to achieve the best trade-off between the patients’ waiting times and the over-utilization of surgical resources. Considering the curses of dimensionality resulting from the large scale of realistically sized problems, we first analyze the structural properties of the MDP and propose an algorithm that facilitates the search for best actions. We then develop a novel reinforcement-learning-based ADP algorithm as the solution technique. Experimental results reveal that the proposed algorithms consume much less computation time in comparison with that required by conventional dynamic programming methods. Additionally, the algorithms are shown to be capable of computing high-quality near-optimal policies for realistically sized problems.

在本文中，我们提出了一种近似动态规划（ADP）算法来解决马尔科夫决策过程（MDP）公式的选择性患者入院控制。为了公平，有效地管理多个专业的择期患者，我们建立了一个等待名单，并为每位患者分配了一个与时间相关的动态优先级评分。然后，考虑到患者的随机到达，每周进行一次顺序决策。在每周结束时，我们从候补名单中选择要在下周接受治疗的患者。通过最小化无限期MDP的成本功能，我们寻求在患者的等待时间与手术资源的过度利用之间取得最佳平衡。考虑到因实际问题规模庞大而引起的尺寸诅咒，我们首先分析了MDP的结构特性，并提出了一种算法，该算法可促进最佳动作的搜索。然后，我们开发了一种新的基于增强学习的ADP算法作为求解技术。实验结果表明，与传统的动态编程方法相比，该算法消耗的计算时间少得多。此外，算法显示出能够针对实际大小的问题计算出高质量的接近最优的策略。实验结果表明，与传统的动态规划方法相比，该算法消耗的计算时间少得多。此外，算法显示出能够针对实际大小的问题计算出高质量的接近最优的策略。实验结果表明，与传统的动态规划方法相比，该算法消耗的计算时间少得多。此外，算法显示出能够针对实际大小的问题计算出高质量的接近最优的策略。