Background 

Surgery for thumb carpometacarpal osteoarthritis is offered to patients who do not benefit from nonoperative treatment. Although surgery is generally successful in reducing symptoms, not all patients benefit. Predicting clinical improvement after surgery could provide decision support and enhance preoperative patient selection.

Questions/purposes 

This study aimed to develop and validate prediction models for clinically important improvement in (1) pain and (2) hand function 12 months after surgery for thumb carpometacarpal osteoarthritis.

Methods 

Between November 2011 and June 2020, 2653 patients were surgically treated for thumb carpometacarpal osteoarthritis. Patient-reported outcome measures were used to preoperatively assess pain, hand function, and satisfaction with hand function, as well as the general mental health of patients and mindset toward their condition. Patient characteristics, medical history, patient-reported symptom severity, and patient-reported mindset were considered as possible predictors. Patients who had incomplete Michigan Hand outcomes Questionnaires at baseline or 12 months postsurgery were excluded, as these scores were used to determine clinical improvement. The Michigan Hand outcomes Questionnaire provides subscores for pain and hand function. Scores range from 0 to 100, with higher scores indicating less pain and better hand function. An improvement of at least the minimum clinically important difference (MCID) of 14.4 for the pain score and 11.7 for the function score were considered “clinically relevant.” These values were derived from previous reports that provided triangulated estimates of two anchor-based and one distribution-based MCID. Data collection resulted in a dataset of 1489 patients for the pain model and 1469 patients for the hand function model. The data were split into training (60%), validation (20%), and test (20%) dataset. The training dataset was used to select the predictive variables and to train our models. The performance of all models was evaluated in the validation dataset, after which one model was selected for further evaluation. Performance of this final model was evaluated on the test dataset. We trained the models using logistic regression, random forest, and gradient boosting machines and compared their performance. We chose these algorithms because of their relative simplicity, which makes them easier to implement and interpret. Model performance was assessed using discriminative ability and qualitative visual inspection of calibration curves. Discrimination was measured using area under the curve (AUC) and is a measure of how well the model can differentiate between the outcomes (improvement or no improvement), with an AUC of 0.5 being equal to chance. Calibration is a measure of the agreement between the predicted probabilities and the observed frequencies and was assessed by visual inspection of calibration curves. We selected the model with the most promising performance for clinical implementation (that is, good model performance and a low number of predictors) for further evaluation in the test dataset.

Results 

For pain, the random forest model showed the most promising results based on discrimination, calibration, and number of predictors in the validation dataset. In the test dataset, this pain model had a poor AUC (0.59) and poor calibration. For function, the gradient boosting machine showed the most promising results in the validation dataset. This model had a good AUC (0.74) and good calibration in the test dataset. The baseline Michigan Hand outcomes Questionnaire hand function score was the only predictor in the model. For the hand function model, we made a web application that can be accessed via https://analyse.equipezorgbedrijven.nl/shiny/cmc1-prediction-model-Eng/.

Conclusion 

We developed a promising model that may allow clinicians to predict the chance of functional improvement in an individual patient undergoing surgery for thumb carpometacarpal osteoarthritis, which would thereby help in the decision-making process. However, caution is warranted because our model has not been externally validated. Unfortunately, the performance of the prediction model for pain is insufficient for application in clinical practice.

Level of Evidence 

Level III, therapeutic study.

中文翻译：

---

机器学习可用于预测拇指腕掌骨关节炎手术后的功能，但不能预测疼痛

背景 

拇指腕掌骨关节炎的手术是为那些无法从非手术治疗中获益的患者提供的。尽管手术通常可以成功减轻症状，但并非所有患者都能受益。预测术后临床改善可以提供决策支持并增强术前患者选择。

问题/目的 

本研究旨在开发和验证拇指腕掌骨关节炎手术后 12 个月（1）疼痛和（2）手功能的临床重要改善的预测模型。

方法 

2011年11月至2020年6月期间，有2653名患者因拇指腕掌骨关节炎接受了手术治疗。患者报告的结果测量用于术前评估疼痛、手功能和手功能满意度，以及患者的一般心理健康状况和对其病情的心态。患者特征、病史、患者报告的症状严重程度和患者报告的心态被认为是可能的预测因素。基线时或术后 12 个月密歇根手结果问卷不完整的患者被排除在外，因为这些分数用于确定临床改善。密歇根手部结果问卷提供了疼痛和手部功能的分项评分。分数范围从 0 到 100，分数越高表示疼痛越轻，手部功能越好。疼痛评分的最小临床重要差异 (MCID) 至少改善 14.4，功能评分改善至少 11.7，则被认为“具有临床相关性”。这些值源自之前的报告，该报告提供了两个基于锚点和一个基于分布的 MCID 的三角估计。数据收集产生了 1489 名患者的疼痛模型数据集和 1469 名患者的手功能模型数据集。数据分为训练数据集 (60%)、验证数据集 (20%) 和测试数据集 (20%)。训练数据集用于选择预测变量并训练我们的模型。在验证数据集中评估所有模型的性能，然后选择一个模型进行进一步评估。最终模型的性能在测试数据集上进行了评估。我们使用逻辑回归、随机森林和梯度增强机训练模型并比较它们的性能。我们选择这些算法是因为它们相对简单，这使得它们更容易实现和解释。使用辨别能力和校准曲线的定性目视检查来评估模型性能。区分度是使用曲线下面积 (AUC) 来衡量的，它衡量模型区分结果（改善或无改善）的能力，AUC 为 0.5 等于机会。校准是预测概率与观察到的频率之间一致性的度量，并通过校准曲线的目视检查进行评估。我们选择了最有希望用于临床实施的模型（即良好的模型性能和少量的预测变量），以便在测试数据集中进行进一步评估。

结果 

对于疼痛，随机森林模型根据验证数据集中的辨别、校准和预测变量数量显示了最有希望的结果。在测试数据集中，该疼痛模型的 AUC (0.59) 和校准较差。对于功能而言，梯度增强机在验证数据集中显示出了最有希望的结果。该模型在测试数据集中具有良好的 AUC (0.74) 和良好的校准。基线密歇根手结果问卷手功能评分是模型中唯一的预测因子。对于手功能模型，我们制作了一个 Web 应用程序，可以通过https://analysis.equipezorgbedrijven.nl/shiny/cmc1-prediction-model-Eng/访问。

结论 

我们开发了一个有前途的模型，可以让临床医生预测接受拇指腕掌骨关节炎手术的个体患者功能改善的机会，从而有助于决策过程。然而，需要谨慎，因为我们的模型尚未经过外部验证。不幸的是，疼痛预测模型的性能不足以应用于临床实践。

证据水平 

III级，治疗研究。