The recent availability of large datasets in bio-medicine has inspired the development of representation learning methods for multiple healthcare applications. Despite advances in predictive performance, the clinical utility of such methods is limited when exposed to real-world data. Here we develop model diagnostic measures to detect potential pitfalls during deployment without assuming access to external data. Specifically, we focus on modeling realistic data shifts in electrophysiological signals (EEGs) via data transforms, and extend the conventional task-based evaluations with analyses of a) model's latent space and b) predictive uncertainty, under these transforms. We conduct experiments on multiple EEG feature encoders and two clinically relevant downstream tasks using publicly available large-scale clinical EEGs. Within this experimental setting, our results suggest that measures of latent space integrity and model uncertainty under the proposed data shifts may help anticipate performance degradation during deployment.

中文翻译：

---

通过潜在空间和不确定性分析评估 EEG 表示在数据转移下的鲁棒性

最近生物医学中大型数据集的可用性激发了用于多种医疗保健应用的表示学习方法的开发。尽管在预测性能方面取得了进步，但当接触到真实世界的数据时，这种方法的临床效用是有限的。在这里，我们开发模型诊断措施来检测部署期间的潜在陷阱，而不假设访问外部数据。具体来说，我们专注于通过数据转换对电生理信号 (EEG) 中的真实数据变化进行建模，并通过在这些转换下分析 a) 模型的潜在空间和 b) 预测不确定性来扩展传统的基于任务的评估。我们使用公开可用的大规模临床脑电图对多个脑电图特征编码器和两个临床相关的下游任务进行实验。