Background: Type 2 diabetes mellitus (T2DM) and its related complications represent a growing economic burden for many countries and health systems. Diabetes complications can be prevented through better disease control, but there is a large gap between the recommended treatment and the treatment that patients actually receive. The treatment of T2DM can be challenging because of different comprehensive therapeutic targets and individual variability of the patients, leading to the need for precise, personalized treatment. Objective: The aim of this study was to develop treatment recommendation models for T2DM based on deep reinforcement learning. A retrospective analysis was then performed to evaluate the reliability and effectiveness of the models. Methods: The data used in our study were collected from the Singapore Health Services Diabetes Registry, encompassing 189,520 patients with T2DM, including 6,407,958 outpatient visits from 2013 to 2018. The treatment recommendation model was built based on 80% of the dataset and its effectiveness was evaluated with the remaining 20% of data. Three treatment recommendation models were developed for antiglycemic, antihypertensive, and lipid-lowering treatments by combining a knowledge-driven model and a data-driven model. The knowledge-driven model, based on clinical guidelines and expert experiences, was first applied to select the candidate medications. The data-driven model, based on deep reinforcement learning, was used to rank the candidates according to the expected clinical outcomes. To evaluate the models, short-term outcomes were compared between the model-concordant treatments and the model-nonconcordant treatments with confounder adjustment by stratification, propensity score weighting, and multivariate regression. For long-term outcomes, model-concordant rates were included as independent variables to evaluate if the combined antiglycemic, antihypertensive, and lipid-lowering treatments had a positive impact on reduction of long-term complication occurrence or death at the patient level via multivariate logistic regression. Results: The test data consisted of 36,993 patients for evaluating the effectiveness of the three treatment recommendation models. In 43.3% of patient visits, the antiglycemic medications recommended by the model were concordant with the actual prescriptions of the physicians. The concordant rates for antihypertensive medications and lipid-lowering medications were 51.3% and 58.9%, respectively. The evaluation results also showed that model-concordant treatments were associated with better glycemic control (odds ratio [OR] 1.73, 95% CI 1.69-1.76), blood pressure control (OR 1.26, 95% CI, 1.23-1.29), and blood lipids control (OR 1.28, 95% CI 1.22-1.35). We also found that patients with more model-concordant treatments were associated with a lower risk of diabetes complications (including 3 macrovascular and 2 microvascular complications) and death, suggesting that the models have the potential of achieving better outcomes in the long term. Conclusions: Comprehensive management by combining knowledge-driven and data-driven models has good potential to help physicians improve the clinical outcomes of patients with T2DM; achieving good control on blood glucose, blood pressure, and blood lipids; and reducing the risk of diabetes complications in the long term.

This is the abstract only. Read the full article on the JMIR site. JMIR is the leading open access journal for eHealth and healthcare in the Internet age.

中文翻译：

---

使用强化学习的 2 型糖尿病管理的有效治疗建议：治疗建议模型开发和验证

背景：2 型糖尿病 (T2DM) 及其相关并发症给许多国家和卫生系统带来了越来越大的经济负担。糖尿病并发症可以通过更好的疾病控制来预防，但推荐的治疗与患者实际接受的治疗之间存在很大差距。由于不同的综合治疗目标和患者的个体差异，T2DM 的治疗可能具有挑战性，导致需要精确、个性化的治疗。目的：本研究的目的是开发基于深度强化学习的 T2DM 治疗推荐模型。然后进行回顾性分析以评估模型的可靠性和有效性。方法：我们研究中使用的数据来自新加坡健康服务糖尿病登记处，包括 189,520 名 T2DM 患者，其中包括 2013 年至 2018 年的 6,407,958 次门诊就诊。治疗推荐模型基于 80% 的数据集构建，其有效性评估为剩下的 20% 的数据。通过结合知识驱动模型和数据驱动模型，开发了三种用于降糖、抗高血压和降脂治疗的治疗推荐模型。基于临床指南和专家经验的知识驱动模型首先应用于选择候选药物。基于深度强化学习的数据驱动模型用于根据预期的临床结果对候选人进行排名。为了评估模型，通过分层、倾向评分加权和多变量回归，通过混杂因素调整，比较了模型一致治疗和模型非一致治疗之间的短期结果。对于长期结果，模型一致率作为自变量包括在内，以评估联合降糖、抗高血压和降脂治疗是否通过多变量逻辑对减少患者水平的长期并发症发生或死亡产生积极影响回归。结果：测试数据包括 36,993 名患者，用于评估三种治疗推荐模型的有效性。在 43.3% 的患者就诊中，模型推荐的降糖药物与医生的实际处方一致。降压药和降脂药的一致率分别为51.3%和58.9%。评估结果还表明，模型一致的治疗与更好的血糖控制（优势比 [OR] 1.73，95% CI 1.69-1.76）、血压控制（OR 1.26，95% CI，1.23-1.29）和血脂质对照（OR 1.28，95% CI 1.22-1.35）。我们还发现，接受更多模型一致治疗的患者糖尿病并发症（包括 3 种大血管并发症和 2 种微血管并发症）和死亡风险较低，这表明这些模型有可能在长期内取得更好的结果。结论：知识驱动和数据驱动相结合的综合管理模式具有帮助医生改善T2DM患者临床预后的良好潜力；血糖、血压、血脂控制良好；并长期降低糖尿病并发症的风险。

这只是摘要。阅读 JMIR 网站上的完整文章。JMIR 是互联网时代电子健康和医疗保健领域领先的开放获取期刊。