Multi-channel lung sound classification with convolutional recurrent neural networks
Introduction
Commercially available lung sound recording devices, i.e. electronic stethoscopes, are limited to single-channel auscultation/recording. In lung auscultation, abnormal/adventitious lung sounds are heard over affected areas, hence, several successive recording positions over the chest are needed to cover the whole organ. As an example, adventitious sounds caused by early stage idiopathic pulmonary fibrosis (IPF) are fine (or velcro) crackles during inspiration, heard over basal areas of the lung [1], [2]. Simultaneous (multi-channel) recording at several positions provides additional information, especially in combination with computer-aided lung sound analysis, which increases the potential of acoustic lung diagnostic.
Several approaches to multi-channel lung sound classification exist. Several research groups record lung sounds independently with different recording setups, i.e. differing in design, and number and position of sensors. A first approach to multi-channel lung sound analysis was the STG16 [3]. It enables 14-channel lung sound recording on the posterior chest, with two additional channels for the locations trachea and heart. Algorithms enable the detection and localization of different adventitious sounds. Another multi-channel recording device with 14-channel lung sound recording on the posterior chest, however, with a different sensor arrangement than the STG16 [3], is presented in [4]. The authors of [4] explore a useful methodology for the classification of the three-class structure (healthy-obstructive–restrictive) in [5]. They model 14-channel pulmonary sound data using a second order vector autoregressive (VAR) model, and feed the estimated model parameter to support vector machine (SVM) and Gaussian mixture model (GMM) classifiers. A 25-channel lung sound recording device is used in [6], with a 5 × 5 sensor array attached on the posterior chest. The authors assess different parameterization techniques for multi-channel lung sounds for two-class classification (normal versus abnormal), such as power spectral density (PSD), the eigenvalues of the covariance matrix, the univariate autoregressive model (UAR), and the multivariate autoregressive model (MAR). Those methods are applied to construct feature vectors used as input to a supervised multilayer neural network. Furthermore, the respiratory sound database from the ICBHI 2017 Challenge also partially consists of multi-channel lung sound recordings [7]. These multi-channel recordings were collected with either seven stethoscopes (3M Littman Classic II SE) with a microphone in the main tube or seven air-coupled electret microphones (C 417 PP, AKG Acoustics) located into capsules made of Teflon. Whether single- or multi-channel processing, three different levels of adventitious sound analysis can be performed [8]: Detection and classification of adventitious sounds at a segment level (i.e. by signal windowing segments are generated, features are extracted, and with random segments of adventitious and normal sounds classification is performed), classification at the event level (of manually isolated events of adventitious and normal lung sounds), and event detection at recording level. Within this work, we focus on the classification of isolated (multi-channel) lung sound recordings with one full breathing cycle each.
Multilayer perceptrons (MLPs), also called feedforward neural networks (FNNs), are widely used for lung sound classification [6], [9], [10]. Although MLPs are very powerful, they do not model temporal context explicitly. To process sequential input of variable length and learn temporal dependencies within the data, recurrent neural networks (RNNs) are suitable architectures [11], [12]. They show state-of-the-art performance in several audio classification tasks, such as speech recognition [12], acoustic event detection and scene classification [13], heart sound classification [14], and are already introduced to event detection in and classification of lung sounds [15], [16], [17], [18]. Another powerful neural network architecture are convolutional neural networks [19]. They are widely applied to audio classification tasks, including lung sound classification [20], [21]. Convolutional neural networks can be used as feature extractors by directly applying them to raw audio waveforms [22], [23]. Another approach is the usage after feature extraction, e.g. by processing spectrograms [24].
In this paper, we exploit spectral, temporal and spatial information for multi-channel lung sounds classification. To this end, we present a multi-channel lung sound classification framework with convolutional recurrent neural networks (CRNN) [24], [25], a combination of convolutional neural networks (CNNs) [19] and recurrent neural networks (RNNs) [11], [12]. For this purpose, we firstly conducted a clinical trial to record a lung sound database with our recently developed multi-channel lung sound recording device [26]. The device enables the recording of lung sounds at 16 positions over the posterior chest. The fixed pattern for the lung sound transducer arrangement of the recording front-end results in varying recording positions depending on the subjects physique. Our proposed classification framework is an approach to render exact recording positions dispensable. We evaluate the proposed method for the diagnosis of idiopathic pulmonary fibrosis in our experiments. The simplified overall processing framework is illustrated in Fig. 1.
Our main contributions and results can be summarized as follows:
- •
For the first time, we introduce a classification framework with CRNNs for multi-channel lung sound recordings. In particular, we present a specific network architecture, which conveniently allows to exploit spectral, temporal, and spatial (i.e. multi-channel) information from multi-channel lung sounds.
- •
We conducted a clinical trial to record a multi-channel lung sound database with healthy and pathological (i.e. idiopathic pulmonary fibrosis) subjects.
- •
We present experimental results, where we compare different neural network architectures for classification.
- •
Together with our multi-channel lung sound recording device, we present a holistic approach to multi-channel lung sound analysis/classification.
The paper is structured as follows: In Section 2, we discuss different neural network architectures, including MLPs, RNNs, and CNNs. In Section 3, we present our custom-build multi-channel lung sound recording device. In Section 4, we present our proposed multi-channel classification framework, the recorded multi-channel lung sound database, the experimental setup (including the evaluation metrics), and the experimental results. Finally, we discuss our findings in Section 5 and conclude the paper in Section 6.
Section snippets
Multilayer perceptron
Multilayer perceptrons (MLPs) [27] are the simplest type of artificial neural networks. In an MLP, information flows forward through the network, i.e. the output of the model is not fed back into itself. A special kind of MLPs are CNNs (see Section 2.3). Extensions of MLPs with feedback connections are RNNs (see Section 2.2).
Eqs. (1)–(2) describe the MLP mathematically. It consists of several layers , with being the index of the hidden layers.
Multi-channel lung sound recording device
With the endeavor of bringing automatic lung sound analysis a step closer to clinical practice, we started with the development of a lung sound recording device [26]. We revealed several limitations in existing methods for (multi-channel) lung sound recording hardware, resulting in the following aspects mainly influencing our hardware design: ease of use, high signal quality, robustness against air- and body-borne noise, multi-channel recording, and airflow-awareness.1
Multi-channel classification framework
The proposed classification framework processes multi-channel lung sound recordings of one breathing cycle each.
Discussion
In our experiments, we compare different neural network architectures for multi-channel lung sound classification. Firstly, we determine a suitable network size for each architecture using grid search. We compare the architectures of the MLP, the BiGRNN, and the ConvBiGRNN, with the latter outperforming the rest.
As initially described (see Section 2), adventitious sounds caused by IPF are inspiratory fine (or velcro) crackles heard over affected areas [1], [2]. Because adventitious sounds are
Conclusion
In this paper, we introduce convolutional recurrent neural networks to multi-channel lung sound classification. To this end, we recorded a small lung sound database with our recently developed multi-channel lung sound recording device, including lung-healthy subjects and patients diagnosed with idiopathic pulmonary fibrosis (IPF). With the acquired data, we perform experiments to evaluate the classification performance of our proposed method, including the comparison with different neural
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This project was supported by the government of Styria, Austria, under the project call HTI:Tech for Med and by the Austrian Science Fund (FWF) under the project number P27803-N15.
We acknowledge 3MTM for providing Littmann® stethoscope chest pieces, Schiller AG for the support with a spirometry solution, and NVIDIA for providing GPU computing resources. We thank the Clinical Trial Coordination Centre of the Medical University of Graz for project management support and monitoring.
References (47)
- et al.
Assessment of multichannel lung sounds parameterization for two-class classification in interstitial lung disease patients
Comput. Biol. Med.
(2011) - et al.
Two-stage classification of respiratory sound patterns
Comput. Biol. Med.
(2005) - et al.
The attractor recurrent neural network based on fuzzy functions: An effective model for the classification of lung abnormalities
Comput. Biol. Med.
(2017) - et al.
Measurement of respiratory acoustical signals. comparison of sensors.
CHEST J.
(1993) - et al.
Measurement of respiratory acoustic signals. effect of microphone air Cavity width, shape, and venting
CHEST J.
(1995) - et al.
Transmission of crackles in patients with interstitial pulmonary fibrosis, congestive heart failure, and pneumonia
CHEST J.
(2005) - et al.
Fundamentals of lung auscultation
New Engl. J. Med.
(2014) - et al.
Automated analysis of crackles in patients with interstitial pulmonary fibrosis
Pulm. Med.
(2011) Computerized multichannel lung sound analysis
IEEE Eng. Med. Biol. Mag.
(2007)- I. Sen, Y. Kahya, A multi-channel device for respiratory sound data acquisition and transient detection, in:...
A comparison of SVM and GMM-based classifier configurations for diagnostic classification of pulmonary sounds
IEEE Trans. Biomed. Eng.
A respiratory sound database for the development of automated classification
Automatic adventitious respiratory sound analysis: A systematic review
PLoS One
Artificial neural networks for acoustic lung signals classification
Sequence to sequence learning with neural networks
Hybrid speech recognition with deep bidirectional LSTM
Gated recurrent networks applied to acoustic scene classification and acoustic event detection
IEEE AASP Chall.: Detect. Classif. Acoustic Scenes Events
Heart sound segmentation-an event detection approach using deep recurrent neural networks
IEEE Trans. Biomed. Eng.
Noise masking recurrent neural network for respiratory sound classification
Deep auscultation: Predicting respiratory anomalies and diseases via recurrent neural networks
Convolutional networks for images, speech, and time series
Handb. Brain Theory Neural Netw.
Classification of lung sounds using convolutional neural networks
EURASIP J. Image Video Process.
Cited by (55)
DeepRespNet: A deep neural network for classification of respiratory sounds
2024, Biomedical Signal Processing and ControlTRespNET: A dual-route exploratory CNN model for pediatric adventitious respiratory sound identification
2024, Biomedical Signal Processing and ControlOpen-set lung sound recognition model based on conditional Gaussian capsule network and variational time–frequency feature reconstruction
2024, Biomedical Signal Processing and ControlBRACETS: Bimodal repository of auscultation coupled with electrical impedance thoracic signals
2023, Computer Methods and Programs in BiomedicineClassification of nucleic acid amplification on ISFET arrays using spectrogram-based neural networks
2023, Computers in Biology and Medicine