Elsevier

Applied Soft Computing

Volume 109, September 2021, 107522
Applied Soft Computing

A deep-learning based multimodal system for Covid-19 diagnosis using breathing sounds and chest X-ray images

https://doi.org/10.1016/j.asoc.2021.107522Get rights and content

Highlights

  • This paper proposes Ai-CovScan, a multimodal deep-learning framework for detecting Covid-19.

  • This framework comprises a transfer-learning-based CovScanNet, where CNN-output is fed into an MLP.

  • Breathing sound spectrograms are analysed for abnormalities using CovScanNet for Covid-19 prognosis.

  • A chest X-ray image analysis is also performed for detecting Covid-19 using a curated dataset.

  • The system is implemented using a smartphone application for rapid detection of Covid-19.

Abstract

Covid-19 has become a deadly pandemic claiming more than three million lives worldwide. SARS-CoV-2 causes distinct pathomorphological alterations in the respiratory system, thereby acting as a biomarker to aid its diagnosis. A multimodal framework (Ai-CovScan) for Covid-19 detection using breathing sounds, chest X-ray (CXR) images, and rapid antigen test (RAnT) is proposed. Transfer Learning approach using existing deep-learning Convolutional Neural Network (CNN) based on Inception-v3 is combined with Multi-Layered Perceptron (MLP) to develop the CovScanNet model for reducing false-negatives. This model reports a preliminary accuracy of 80% for the breathing sound analysis, and 99.66% Covid-19 detection accuracy for the curated CXR image dataset. Based on Ai-CovScan, a smartphone app is conceptualised as a mass-deployable screening tool, which could alter the course of this pandemic. This app’s deployment could minimise the number of people accessing the limited and expensive confirmatory tests, thereby reducing the burden on the severely stressed healthcare infrastructure.

Keywords

Covid-19
CNN
MLP
Chest X-ray images
Breathing sounds
Deep-learning

Cited by (0)

1

The contribution of this author is equivalent to first author.

View Abstract