In precision medicine, the ultimate goal is to recommend the most effective treatment to an individual patient based on patient-specific molecular and clinical profiles, possibly high-dimensional. To advance cancer treatment, large-scale screenings of cancer cell lines against chemical compounds have been performed to help better understand the relationship between genomic features and drug response; existing machine learning approaches use exclusively supervised learning, including penalized regression and recommender systems. However, it would be more efficient to apply reinforcement learning to sequentially learn as data accrue, including selecting the most promising therapy for a patient given individual molecular and clinical features and then collecting and learning from the corresponding data. In this article, we propose a novel personalized ranking system called Proximal Policy Optimization Ranking (PPORank), which ranks the drugs based on their predicted effects per cell line (or patient) in the framework of deep reinforcement learning (DRL). Modeled as a Markov decision process, the proposed method learns to recommend the most suitable drugs sequentially and continuously over time. As a proof-of-concept, we conduct experiments on two large-scale cancer cell line data sets in addition to simulated data. The results demonstrate that the proposed DRL-based PPORank outperforms the state-of-the-art competitors based on supervised learning. Taken together, we conclude that novel methods in the framework of DRL have great potential for precision medicine and should be further studied.

中文翻译：

---

深度强化学习用于个性化治疗推荐


在精准医学中，最终目标是根据患者特定的分子和临床特征（可能是高维的）向个体患者推荐最有效的治疗方法。为了推进癌症治疗，针对化合物对癌细胞系进行了大规模筛查，以帮助更好地了解基因组特征与药物反应之间的关系；现有的机器学习方法仅使用监督学习，包括惩罚回归和推荐系统。然而，应用强化学习随着数据的积累而顺序学习会更有效，包括根据个体分子和临床特征为患者选择最有希望的治疗方法，然后收集并从相应的数据中学习。在本文中，我们提出了一种称为近端策略优化排名（PPORank）的新型个性化排名系统，该系统在深度强化学习（DRL）框架中根据每个细胞系（或患者）的预测效果对药物进行排名。所提出的方法被建模为马尔可夫决策过程，学会随着时间的推移顺序和连续地推荐最合适的药物。作为概念验证，除了模拟数据之外，我们还对两个大规模癌细胞系数据集进行了实验。结果表明，所提出的基于 DRL 的 PPORank 优于基于监督学习的最先进的竞争对手。综上所述，我们得出的结论是，DRL 框架内的新方法在精准医学方面具有巨大潜力，值得进一步研究。