To enhance deep learning-based automated interictal epileptiform discharge (IED) detection, this study proposes a multimodal method, vEpiNet, that leverages video and electroencephalogram (EEG) data. Datasets comprise 24 931 IED (from 484 patients) and 166 094 non-IED 4-second video-EEG segments. The video data is processed by the proposed patient detection method, with frame difference and Simple Keypoints (SKPS) capturing patients’ movements. EEG data is processed with EfficientNetV2. The video and EEG features are fused via a multilayer perceptron. We developed a comparative model, termed nEpiNet, to test the effectiveness of the video feature in vEpiNet. The 10-fold cross-validation was used for testing. The 10-fold cross-validation showed high areas under the receiver operating characteristic curve (AUROC) in both models, with a slightly superior AUROC (0.9902) in vEpiNet compared to nEpiNet (0.9878). Moreover, to test the model performance in real-world scenarios, we set a prospective test dataset, containing 215 h of raw video-EEG data from 50 patients. The result shows that the vEpiNet achieves an area under the precision–recall curve (AUPRC) of 0.8623, surpassing nEpiNet’s 0.8316. Incorporating video data raises precision from 70% (95% CI, 69.8%–70.2%) to 76.6% (95% CI, 74.9%–78.2%) at 80% sensitivity and reduces false positives by nearly a third, with vEpiNet processing one-hour video-EEG data in 5.7 min on average. Our findings indicate that video data can significantly improve the performance and precision of IED detection, especially in prospective real clinic testing. It suggests that vEpiNet is a clinically viable and effective tool for IED analysis in real-world applications.

中文翻译：

---

vEpiNet：基于视频和脑电图数据的多模态癫痫样放电检测方法

为了增强基于深度学习的自动发作间期癫痫样放电 (IED) 检测，本研究提出了一种利用视频和脑电图 (EEG) 数据的多模态方法 vEpiNet。数据集包括 24 931 个 IED（来自 484 名患者）和 166 094 个非 IED 4 秒视频脑电图片段。视频数据通过所提出的患者检测方法进行处理，通过帧差和简单关键点（SKPS）捕获患者的运​​动。 EEG 数据使用 EfficientNetV2 进行处理。视频和脑电图特征通过多层感知器融合。我们开发了一个名为 nEpiNet 的比较模型来测试 vEpiNet 中视频特征的有效性。使用10倍交叉验证进行测试。 10 倍交叉验证显示，两个模型的受试者工作特征曲线下面积 (AUROC) 都较高，其中 vEpiNet 的 AUROC (0.9902) 略优于 nEpiNet (0.9878)。此外，为了测试模型在现实场景中的性能，我们设置了一个前瞻性测试数据集，其中包含来自 50 名患者的 215 小时原始视频脑电图数据。结果表明，vEpiNet 的精确召回曲线下面积 (AUPRC) 为 0.8623，超过了 nEpiNet 的 0.8316。结合视频数据可将灵敏度从 70%（95% CI，69.8%–70.2%）提高到 76.6%（95% CI，74.9%–78.2%），灵敏度为 80%，并将误报率减少近三分之一，其中 vEpiNet 处理一个平均 5.7 分钟内的 1 小时视频脑电图数据。我们的研究结果表明，视频数据可以显着提高简易爆炸装置检测的性能和精度，特别是在前瞻性真实临床测试中。它表明 vEpiNet 是一种临床上可行且有效的工具，可用于实际应用中的 IED 分析。