Targeting the uncertainty of predictions at patient-level using an ensemble of classifiers coupled with calibration methods, Venn-ABERS, and Conformal Predictors: A case study in AD.,Journal of Biomedical informatics

当前位置： X-MOL 学术 › J. Biomed. Inform. › 论文详情

Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)

Targeting the uncertainty of predictions at patient-level using an ensemble of classifiers coupled with calibration methods, Venn-ABERS, and Conformal Predictors: A case study in AD.
Journal of Biomedical informatics ( IF 4.0 ) Pub Date : 2019-12-06 , DOI: 10.1016/j.jbi.2019.103350
Telma Pereira ₁ , Sandra Cardoso ₂ , Manuela Guerreiro ₂ , Alexandre Mendonça ₂ , Sara C Madeira ₃ ,

Affiliation

Despite being able to make accurate predictions, most existing prognostic models lack a proper indication about the uncertainty of each prediction, that is, the risk of prediction error for individual patients. This hampers their translation to primary care settings through decision support systems. To address this problem, we studied different methods for transforming classifiers into probabilistic/confidence-based predictors (here called uncertainty methods), where predictions are complemented with probability estimates/confidence regions reflecting their uncertainty (uncertainty estimates). We tested several uncertainty methods: two well-known calibration methods (Platt Scaling and Isotonic Regression), Conformal Predictors, and Venn-ABERS predictors. We evaluated whether these methods produce valid predictions, where uncertainty estimates reflect the ground truth probabilities. Furthermore, we assessed the proportion of valid predictions made at high-certainty thresholds (predictions with uncertainty measures above a given threshold) since this impacts their usefulness in clinical decisions. Finally, we proposed an ensemble-based approach where predictions from multiple pairs of (classifier, uncertainty method) are combined to predict whether a given MCI patient will convert to AD. This ensemble should putatively provide predictions for a larger number of patients while releasing users from deciding which pair of (classifier, uncertainty method) is more appropriate for data under study. The analysis was performed with a Portuguese cohort (CCC) of around 400 patients and validated in the publicly available ADNI cohort. Despite our focus on MCI to AD prognosis, the proposed approach can be applied to other diseases and prognostic problems.

中文翻译：

使用分类器与校准方法，Venn-ABERS和保形预测器的组合将患者的预测不确定性作为目标，以AD为例。

尽管能够做出准确的预测，但是大多数现有的预后模型都缺乏关于每个预测的不确定性的正确指示，即每个患者的预测错误的风险。这会阻碍他们通过决策支持系统将其转换为初级保健环境。为了解决这个问题，我们研究了将分类器转换为基于概率/置信度的预测变量的不同方法（此处称为不确定性方法），其中，预测通过反映其不确定性的概率估计/置信区域（不确定性估计）进行补充。我们测试了几种不确定性方法：两种著名的校准方法（Platt标度和等渗回归），保形预测变量和Venn-ABERS预测变量。我们评估了这些方法是否产生有效的预测，不确定性估计值反映了地面真实概率。此外，我们评估了在高确定性阈值下做出的有效预测的比例（不确定性度量高于给定阈值的预测），因为这会影响其在临床决策中的作用。最后，我们提出了一种基于整体的方法，其中将来自多对（分类器，不确定性方法）的预测结合起来，以预测给定的MCI患者是否会转换为AD。该集合应该可以为更多的患者提供预测，同时让用户不必决定哪对（分类器，不确定性方法）更适合所研究的数据。该分析是在约400名患者的葡萄牙队列（CCC）中进行的，并在可公开获得的ADNI队列中进行了验证。

更新日期：2019-12-06

点击分享查看原文

点击收藏

阅读更多本刊最新论文本刊介绍/投稿指南11