The multi-armed restless bandit framework allows to model a wide variety of decision-making problems in areas as diverse as industrial engineering, computer communication, operations research, financial engineering, communication networks etc. In a seminal work, Whittle developed a methodology to derive well-performing (Whittle’s) index policies that are obtained by solving a relaxed version of the original problem. However, the computation of Whittle’s index itself is a difficult problem and hence researchers focused on calculating Whittle’s index numerically or with a problem dependent approach. In our main contribution we derive an analytical expression for Whittle’s index for any Markovian bandit with both finite and infinite transition rates. We derive sufficient conditions for the optimal solution of the relaxed problem to be of threshold type, and obtain conditions for the bandit to be indexable, a property assuring the existence of Whittle’s index. Our solution approach provides a unifying expression for Whittle’s index, which we highlight by retrieving known indices from literature as particular cases. The applicability of finite rates is illustrated with the machine repairmen problem, and that of infinite rates by an example of communication networks where transmission rates react instantaneously to packet losses.

多臂躁动不安的匪徒框架允许在工业工程，计算机通信，运筹学，金融工程，通信网络等领域建模各种决策问题。在开创性的工作中，Whittle开发了一种方法来推导通过解决原始问题的宽松版本而获得的性能良好的（惠特尔）索引策略。但是，Whittle指数的计算本身是一个难题，因此研究人员着重于通过数值或采用问题相关方法来计算Whittle指数。在我们的主要贡献中，我们导出了具有有限和无限过渡率的任何马尔可夫强盗的惠特尔指数的解析表达式。我们推导了充分的条件，以使松弛问题的最优解成为阈值类型，并获得使强盗可索引的条件，该条件确保了Whittle索引的存在。我们的解决方案方法为Whittle索引提供了统一的表达方式，我们通过从文献中检索特定情况下的已知索引来突出显示该表达方式。机器维修问题说明了有限速率的适用性，而传输速率立即对数据包丢失做出反应的通信网络举例说明了无限速率的适用性。