In this article, we share our experiences in designing and developing a suite of deep neural network–(DNN) based COVID-19 case detection and recognition framework. Existing pathological tests such as RT-PCR-based pathogen RNA detection from nasal swabbing seem to display low detection rates during the early stages of virus contraction. Moreover, the reliance on a few overburdened laboratories based around an epicenter capable of supplying large numbers of RT-PCR tests makes this testing method non-scalable when the rate of infections is high. Similarly, finding an effective drug or vaccine with which to combat COVID-19 requires a long time and many clinical trials. The development of pathological COVID-19 tests is hindered by shortages in the supply chain of chemical reagents necessary for testing on a large scale. This diminishes the speed of diagnosis and the ability to filter out COVID-19 positive patients from uninfected patients on a national level. Existing research has shown that DNN has been successful in identifying COVID-19 from radiological media such as CT scans and X-ray images, audio media such as cough sounds, optical coherence tomography to identify conjunctivitis and pink eye symptoms on the ocular surface, body temperature measurement using smartphone fingerprint sensors or thermal cameras, the use of live facial detection to identify safe social distancing practices from camera images, and face mask detection from camera images. We also investigate the utility of federated learning in diagnosis cases where private data can be trained via edge learning. These point-of-care modalities can be integrated with DNN-based RT-PCR laboratory test results to assimilate multiple modalities of COVID-19 detection and thereby provide more dimensions of diagnosis. Finally, we will present our initial test results, which are encouraging.

中文翻译：

---

用于 COVID-19 诊断的多模式、多媒体床旁深度学习框架

在本文中，我们分享了我们在设计和开发一套基于深度神经网络（DNN）的 COVID-19 病例检测和识别框架方面的经验。现有的病理学测试，例如基于 RT-PCR 的鼻拭子病原体 RNA 检测似乎在病毒收缩的早期阶段显示出低检出率。此外，依赖于几个负担过重的实验室，这些实验室位于能够提供大量 RT-PCR 检测的震中，使得这种检测方法在感染率高时无法扩展。同样，找到一种有效的药物或疫苗来对抗 COVID-19 需要很长时间和许多临床试验。大规模检测所需的化学试剂供应链短缺，阻碍了病理性 COVID-19 检测的发展。这降低了诊断速度以及在全国范围内从未感染患者中筛选出 COVID-19 阳性患者的能力。现有研究表明，DNN 已成功从 CT 扫描和 X 射线图像等放射媒体、咳嗽声等音频媒体、光学相干断层扫描识别眼表、身体上的结膜炎和红眼病症状中识别 COVID-19使用智能手机指纹传感器或热像仪进行温度测量，使用实时面部检测从摄像头图像中识别安全的社交距离做法，以及从摄像头图像中检测面罩。我们还研究了联邦学习在可以通过边缘学习训练私有数据的诊断案例中的效用。这些护理点模式可以与基于 DNN 的 RT-PCR 实验室测试结果相结合，以吸收 COVID-19 检测的多种模式，从而提供更多维度的诊断。最后，我们将展示我们的初步测试结果，这是令人鼓舞的。