Background

The observed consultation length at specialty clinics, such as cardiology care, is represented by two underlying groups - one with zero service time due to patient no-shows, and the other characterized by positive values with high variance. This inconstancy affects the scheduler’s ability to accurately estimate consultation length, which, in turn, hinders effective utilization of the clinic’s resources and timely access to care. The objectives of this study were to: (i) predict the consultation length by accounting for its semicontinuous nature (i.e., zero in case of no-shows and positive otherwise), using machine learning (ML) algorithms, (ii) identify important features for predicting no-shows and non-zero consultation length, and (iii) assess the impact of integrating the ML-based prediction with the appointment scheduling system.

Methods

We used two-years of data extracted from the electronic medical records of a cardiology clinic. By leveraging 16 predictors pertaining to the patient, appointment, and doctor, a two-part ML-based approach was developed to handle the semicontinuous consultation length. Supervised classification models were employed to predict no-shows (i.e., categorize the consultation length as zero or positive), and regression algorithms were developed for estimating non-zero consultation lengths. Three algorithms, namely, random forests, stochastic gradient boosting, and deep neural networks, were individually employed for both no-show classification and positive consultation length prediction. Finally, the best performing classification and regression models were combined to establish the complete two-part model, and its prediction error on new data is benchmarked against the clinic’s current performance. The evaluation metrics for classification models were area under the receiver operating characteristic curve (AUC-ROC) and area under the precision-recall curve (AUC-PR). The prediction performance of regression algorithms was evaluated by mean absolute error (MAE), root mean squared error (RMSE), and mean absolute percentage error (MAPE). A simulation modeling approach was adopted to ascertain the effectiveness of using ML-based prediction for scheduling decisions as opposed to the clinic’s current strategy.

Results

Among the classification models tested, stochastic gradient boosted classification tree (SGBCT) demonstrated best performance (AUC-ROC = 0.85, AUC-PR = 0.64). For positive consultation length prediction, deep neural network regressor (DNNR) resulted in lowest prediction error (MAE = 8.55, RMSE = 6.88, MAPE = 12.24). The complete two-part model (SGBCT + DNNR) outperformed the clinic’s approach to consultation length estimation by achieving 50 % and 52 % reduction in RMSE and MAE, respectively. Further adopting it for appointment scheduling could reduce the patient waiting time and doctor idle time by 56 % and 52 %, respectively. Besides, several clinical insights, along with critical features for no-show and consultation length prediction, were also identified from our proof-of-concept study.

Conclusion

This study demonstrates that routine clinical tasks such as estimation of consultation length and no-shows can be accurately predicted using ML algorithms, and subsequently integrated into the clinical scheduling system to improve resource utilization and reduce patient waiting time.

中文翻译：

---

会诊时长和缺席预测可提高心脏病诊所的约会安排效率：一种数据分析方法

背景

在两个专业小组（例如心脏病学护理）中观察到的咨询时间由两个基本组代表：一个组由于患者未出现而导致服务时间为零，另一个组的特征是正值且差异很大。这种不稳定性会影响调度员准确估计会诊长度的能力，进而阻碍有效利用诊所的资源和及时获得护理。这项研究的目的是：（i）使用机器学习（ML）算法，通过考虑咨询的半连续性质（即，如果没有出现，则为零，否则为肯定）来预测咨询时间，（ii）确定重要特征用于预测未到场和非零咨询时间，以及（iii）评估将基于ML的预测与约会计划系统集成的影响。

方法

我们使用了从心脏病诊所的电子病历中提取的两年数据。通过利用与患者，预约和医生有关的16个预测变量，开发了一种基于ML的两部分方法来处理半连续的咨询时间。监督分类模型被用来预测没有出现（即，咨询长度分类为零或积极），并开发了回归算法来估计非零咨询长度。三种算法，分别是随机森林，随机梯度增强和深度神经网络，分别用于不显示分类和正向咨询长度预测。最后，将效果最好的分类和回归模型相结合，以建立完整的两部分模型，其对新数据的预测误差将根据诊所的当前性能进行基准测试。分类模型的评估指标是接收器工作特性曲线下的面积（AUC-ROC）和精确召回曲线下的面积（AUC-PR）。回归算法的预测性能通过平均绝对误差（MAE），均方根误差（RMSE）和平均绝对百分比误差（MAPE）进行评估。采用模拟建模方法来确定使用基于ML的预测来调度决策的有效性，而不是诊所当前的策略。回归算法的预测性能通过平均绝对误差（MAE），均方根误差（RMSE）和平均绝对百分比误差（MAPE）进行评估。采用模拟建模方法来确定使用基于ML的预测来调度决策的有效性，而不是诊所当前的策略。回归算法的预测性能通过平均绝对误差（MAE），均方根误差（RMSE）和平均绝对百分比误差（MAPE）进行评估。采用模拟建模方法来确定使用基于ML的预测来调度决策的有效性，而不是诊所当前的策略。

结果

在测试的分类模型中，随机梯度增强分类树（SGBCT）表现出最佳性能（AUC-ROC = 0.85，AUC-PR = 0.64）。对于积极的咨询长度预测，深度神经网络回归器（DNNR）导致最低的预测误差（MAE = 8.55，RMSE = 6.88，MAPE = 12.24）。完整的两部分模型（SGBCT + DNNR）通过将RMSE和MAE分别减少50％和52％，优于了诊所的咨询长度估计方法。进一步将其用于约会调度可以分别将患者等待时间和医生闲置时间分别减少56％和52％。此外，从我们的概念验证研究中还发现了一些临床见解，以及未出现和咨询时间预测的关键特征。

结论

这项研究表明，使用ML算法可以准确地预测例行的临床任务，例如，咨询时间的估计和未出现的时间，然后将其集成到临床调度系统中，以提高资源利用率并减少患者的等待时间。