Background and objectives

: The hybrid artificial pancreas regulates glucose levels in people with type 1 diabetes. It delivers (i) insulin boluses at meal times based on the meals’ carbohydrate content and the carbohydrate ratios (CRs) and (ii) insulin basal, between meals and at night, continuously modulated around individual-specific programmed basal rate. The CRs and programmed basal rate significantly vary between individuals and within the same individual with type 1 diabetes, and using suboptimal values in the hybrid artificial pancreas may degrade glucose control. We propose a reinforcement learning algorithm to adaptively optimize CRs and programmed basal rate to improve the performance of the hybrid artificial pancreas.

Methods

: The proposed reinforcement learning algorithm was designed using the Q-learning approach. The algorithm learns the optimal actions (CRs and programmed basal rate) by applying them to the individual's state (previous day's glucose levels and insulin delivery) based on an exploration and exploitation trade-off. First, outcomes from our simulator were compared to those of a clinical study in 23 individuals with type 1 diabetes and have yielded similar results. Second, the learning algorithm was tested using the simulator with two scenarios. Scenario 1 has fixed meal sizes and ingestion times and scenario 2 has a more realistic eating behavior with random meal sizes, ingestion times, and carbohydrate counting errors.

Results

: After about five weeks, the reinforcement learning algorithm improved the percentage of time spent in target range from 67% to 86.7% in scenario 1 and 65.5% to 86% in scenario 2. The percentage of time spent below 4.0 mmol/L decreased from 9% to 0.9% in scenario 1 and 9.5% to 1.1% in scenario 2.

Conclusions

: Results indicate that the proposed algorithm has the potential to improve glucose control in people with type 1 diabetes using the hybrid artificial pancreas. The proposed algorithm is a key in making the hybrid artificial pancreas adaptive for the long-term real life outpatient studies.

中文翻译：

---

通过调节碳水化合物比例和程序化基础速率长期使用混合人工胰腺：强化学习方法

背景和目标

：混合人工胰腺调节1型糖尿病患者的葡萄糖水平。它根据膳食中的碳水化合物含量和碳水化合物比率（CR）来提供（i）进餐时的胰岛素大剂量，以及（ii）膳食之间和晚上的胰岛素基础剂量，围绕个体特定的程序化基础剂量连续调整。在1型糖尿病患者中，个体之间以及同一个体中的CR和程序化基础率显着不同，在混合人工胰腺中使用次优值可能会降低血糖控制。我们提出一种强化学习算法，以自适应地优化CR和编程的基础率，以提高混合人工胰腺的性能。

方法

：提出的强化学习算法是使用Q学习方法设计的。该算法通过在权衡勘探和开发利用的基础上，将最佳操作（CR和已编程的基础速率）应用于个体状态（前一天的葡萄糖水平和胰岛素输送），从而学习最佳操作。首先，我们将模拟器的结果与23位1型糖尿病患者的临床研究结果进行了比较，得出了相似的结果。其次，使用模拟器在两种情况下测试了学习算法。方案1的进餐量和摄入时间固定，方案2的进餐行为较为实际，但进餐量，进食时间和碳水化合物计数错误随机。

结果

：大约五周后，强化学习算法将场景1中目标时间花费的百分比从方案1的67％提高到86.7％，将场景2中目标时间花费的百分比从65.5％提高到86％。低于4.0 mmol / L的时间百分比从在方案1中为9％至0.9％，在方案2中为9.5％至1.1％。

结论

：结果表明，所提出的算法具有使用混合人工胰腺改善1型糖尿病患者血糖控制的潜力。提出的算法是使混合人工胰腺适应长期实际生活门诊研究的关键。