BACKGROUND AND AIMS Clinical staff are typically poor at predicting alcohol dependence treatment outcomes. Machine Learning (ML) offers the potential to model complex clinical data more effectively. This study tested the predictive accuracy of ML algorithms demonstrated to be effective in predicting alcohol dependence outcomes, compared with clinical judgement and traditional linear regression. DESIGN Prospective study. ML models were trained on 1016 previously treated patients (training-set) who attended a hospital-based alcohol and drug clinic. ML models (n=27), clinical psychologists (n=10), and a 'traditional' logistic regression model (n=1) predicted treatment outcome during the initial treatment session of an alcohol dependence program. SETTING A three-month Cognitive Behavioural Therapy (CBT)-based abstinence program for alcohol dependence in a hospital-based alcohol and drug clinic in Australia. PARTICIPANTS Prospective predictions were made for 220 new patients (test-set; 71% male, mean age=35.78 years, sd=9.19). Sixty-nine (31.36%) patients successfully completed treatment. MEASUREMENTS Treatment success was the primary outcome variable. The cross-validated training-set accuracy of ML models was used to determine optimal parameters for selecting models for prospective prediction. Accuracy, sensitivity, specificity, Area Under the receiver operator Curve (AUC), Brier Score, and calibration curves were calculated and compared across predictions. FINDINGS The mean aggregate accuracy of the ML models (63.06%) was higher than the mean accuracy of psychologist predictions (56.36%). The most accurate ML model achieved 70% accuracy, as did logistic regression. Both were more accurate than psychologists (p<.05) and had superior calibration. The high specificity for the selected ML (79%) and logistic regression (90%) meant they were significantly (p<.001) more effective than psychologists (50%) at correctly identifying patients whose treatment was unsuccessful. For ML and logistic regression, high specificity came at the expense of sensitivity (26% and 31% respectively), resulting in poor prediction of successful patients. CONCLUSIONS Machine Learning models and logistic regression appear to be more accurate than psychologists at predicting treatment outcomes in an abstinence program for alcohol dependence, but sensitivity is low.

中文翻译：

---

预测酒精依赖治疗结果：临床心理学家与“训练有素的”机器学习模型的前瞻性比较研究

背景和目的 临床工作人员通常不善于预测酒精依赖治疗的结果。机器学习 (ML) 提供了更有效地模拟复杂临床数据的潜力。与临床判断和传统线性回归相比，这项研究测试了 ML 算法的预测准确性，该算法被证明在预测酒精依赖结果方面是有效的。设计 前瞻性研究。ML 模型在 1016 名先前接受过治疗的患者（训练集）上进行了训练，这些患者在医院酒精和药物诊所就诊。ML 模型 (n=27)、临床心理学家 (n=10) 和“传统”逻辑回归模型 (n=1) 预测了酒精依赖计划初始治疗期间的治疗结果。在澳大利亚的一家医院酒精和药物诊所设置为期三个月的基于认知行为疗法 (CBT) 的酒精依赖戒酒计划。参与者对 220 名新患者（测试集；71% 男性，平均年龄 = 35.78 岁，sd = 9.19）进行了前瞻性预测。69 名（31.36%）患者成功完成治疗。测量 治疗成功是主要结果变量。ML 模型的交叉验证训练集准确性用于确定选择模型进行前瞻性预测的最佳参数。计算准确度、灵敏度、特异性、接受者操作曲线下面积 (AUC)、Brier 分数和校准曲线，并在预测之间进行比较。发现 ML 模型的平均总准确率 (63.06%) 高于心理学家预测的平均准确率 (56.36%)。最准确的 ML 模型达到了 70% 的准确率，逻辑回归也是如此。两者都比心理学家更准确 (p<.05) 并且具有出色的校准。所选 ML (79%) 和逻辑回归 (90%) 的高特异性意味着它们在正确识别治疗失败的患者方面比心理学家 (50%) 更有效 (p<.001)。对于 ML 和逻辑回归，高特异性是以牺牲敏感性为代价的（分别为 26% 和 31%），导致对成功患者的预测不佳。结论 机器学习模型和逻辑回归在预测酒精依赖戒断计划的治疗结果方面似乎比心理学家更准确，但敏感性较低。两者都比心理学家更准确 (p<.05) 并且具有出色的校准。所选 ML (79%) 和逻辑回归 (90%) 的高特异性意味着它们在正确识别治疗失败的患者方面比心理学家 (50%) 更有效 (p<.001)。对于 ML 和逻辑回归，高特异性是以牺牲敏感性为代价的（分别为 26% 和 31%），导致对成功患者的预测不佳。结论 机器学习模型和逻辑回归在预测酒精依赖戒断计划的治疗结果方面似乎比心理学家更准确，但敏感性较低。两者都比心理学家更准确 (p<.05) 并且具有出色的校准。所选 ML (79%) 和逻辑回归 (90%) 的高特异性意味着它们在正确识别治疗失败的患者方面比心理学家 (50%) 更有效 (p<.001)。对于 ML 和逻辑回归，高特异性是以牺牲敏感性为代价的（分别为 26% 和 31%），导致对成功患者的预测不佳。结论 机器学习模型和逻辑回归在预测酒精依赖戒断计划的治疗结果方面似乎比心理学家更准确，但敏感性较低。所选 ML (79%) 和逻辑回归 (90%) 的高特异性意味着它们在正确识别治疗失败的患者方面比心理学家 (50%) 更有效 (p<.001)。对于 ML 和逻辑回归，高特异性是以牺牲敏感性为代价的（分别为 26% 和 31%），导致对成功患者的预测不佳。结论 机器学习模型和逻辑回归在预测酒精依赖戒断计划的治疗结果方面似乎比心理学家更准确，但敏感性较低。所选 ML (79%) 和逻辑回归 (90%) 的高特异性意味着它们在正确识别治疗失败的患者方面比心理学家 (50%) 更有效 (p<.001)。对于 ML 和逻辑回归，高特异性是以牺牲敏感性为代价的（分别为 26% 和 31%），导致对成功患者的预测不佳。结论 机器学习模型和逻辑回归在预测酒精依赖戒断计划的治疗结果方面似乎比心理学家更准确，但敏感性较低。导致对成功患者的预测不佳。结论 机器学习模型和逻辑回归在预测酒精依赖戒断计划的治疗结果方面似乎比心理学家更准确，但敏感性较低。导致对成功患者的预测不佳。结论 机器学习模型和逻辑回归在预测酒精依赖戒断计划的治疗结果方面似乎比心理学家更准确，但敏感性较低。