Intensive Care Unit Electronic Health Records (ICU EHRs) store multimodal data about patients including clinical notes, sparse and irregularly sampled physiological time series, lab results, and more. To date, most methods designed to learn predictive models from ICU EHR data have focused on a single modality. In this paper, we leverage the recently proposed interpolation-prediction deep learning architecture(Shukla and Marlin 2019) as a basis for exploring how physiological time series data and clinical notes can be integrated into a unified mortality prediction model. We study both early and late fusion approaches and demonstrate how the relative predictive value of clinical text and physiological data change over time. Our results show that a late fusion approach can provide a statistically significant improvement in mortality prediction performance over using individual modalities in isolation.

中文翻译：

---

将生理时间序列和临床记录与深度学习相结合，以改进 ICU 死亡率预测

重症监护病房电子病历 (ICU EHR) 存储有关患者的多模式数据，包括临床记录、稀疏和不规则采样的生理时间序列、实验室结果等。迄今为止，大多数旨在从 ICU EHR 数据中学习预测模型的方法都专注于单一模式。在本文中，我们利用最近提出的插值预测深度学习架构（Shukla 和 Marlin 2019）作为探索如何将生理时间序列数据和临床记录整合到统一的死亡率预测模型中的基础。我们研究了早期和晚期融合方法，并展示了临床文本和生理数据的相对预测值如何随时间变化。