Deep learning has demonstrated success in many applications; however, their use in healthcare has been limited due to the lack of transparency into how they generate predictions. Algorithms such as Recurrent Neural Networks (RNNs) when applied to Electronic Medical Records (EMR) introduce additional barriers to transparency because of the sequential processing of the RNN and the multi-modal nature of EMR data. This work seeks to improve transparency by: 1) introducing Learned Binary Masks (LBM) as a method for identifying which EMR variables contributed to an RNN model’s risk of mortality (ROM) predictions for critically ill children; and 2) applying KernelSHAP for the same purpose. Given an individual patient, LBM and KernelSHAP both generate an attribution matrix that shows the contribution of each input feature to the RNN’s sequence of predictions for that patient. Attribution matrices can be aggregated in many ways to facilitate different levels of analysis of the RNN model and its predictions. Presented are three methods of aggregations and analyses: 1) over volatile time periods within individual patient predictions, 2) over populations of ICU patients sharing specific diagnoses, and 3) across the general population of critically ill children.

深度学习已在许多应用中取得了成功；但是，由于缺乏关于如何生成预测的透明度，因此它们在医疗保健中的使用受到限制。由于递归神经网络的顺序处理和EMR数据的多模式性质，诸如递归神经网络（RNN）之类的算法在应用于电子病历（EMR）时为透明度带来了其他障碍。这项工作旨在通过以下方法来提高透明度：1）引入学习型二进制掩码（LBM）作为一种方法，以识别哪些EMR变量有助于RNN模型预测重症儿童的死亡风险（ROM）；和2）出于相同目的应用KernelSHAP。给一个病人 LBM和KernelSHAP都生成一个归因矩阵，该矩阵显示每个输入特征对该患者的RNN预测序列的贡献。归因矩阵可以多种方式汇总，以促进对RNN模型及其预测进行不同级别的分析。提出了三种汇总和分析方法：1）在个体患者预测内的波动时间内，2）在共有特定诊断的ICU患者群体中，以及3）在危重儿童的总体群体中。