Introduction

In December 2019, a viral outbreak, later named COVID-19, occurred in Wuhan, China’s Hubei Province. Due to its high transmission rate, the disease quickly spread throughout the world and caused a pandemic [1]. The number of people infected with the disease exceeded one hundred and seventy million globally, and the number of deaths reached four million by June 2021. Additionally, almost all countries were socially, economically, and politically affected by the epidemic. Although some studies to treat the disease have been conducted, a treatment, drug, or vaccine has not yet been developed. Therefore, it is estimated that the effects of the disease will continue for some time.

COVID-19 is diagnosed through clinical symptoms and blood and biochemical tests [2]. Occasionally, some problems are encountered in its diagnosis because some tests have low sensitivity, throat and nose specimens are sometimes not taken properly, and sometimes the virus has descended from the nose and throat to the lungs in the course of the disease’s progression. In this context, lung and chest radiological images serve as important clinical data for diagnosis and treatment planning.

COVID-19 causes severe pneumonia in its advanced stage and leads to death [3, 4]. Many academic studies [5,6,7,8,9,10] have demonstrated that the disease causes interstitial involvement, lung opacities, bilateral ground-glass opacities, and patchy opacities in the lungs.

Long and Ehrenfeld [11] and McCall [12] stated that the use of artificial intelligence is an inevitable requirement for reducing the effects of the COVID-19 pandemic crisis and the workload of healthcare professionals while increasing and accelerating disease diagnosis and diagnostic success. Many artificial intelligence applications have been created to diagnose COVID-19 using deep learning methods on CT lung images. Table 1 contains detailed information of past studies on COVID-19 and non-COVID-19 classification, and Table 2 includes information on COVID-19 pneumonia and other pneumonia classification.

Table 1 Previous studies on COVID-19 and non-COVID-19 classification
Full size table
Table 2 Previous studies on COVID-19 pneumonia and other pneumonia classification
Full size table

The main purpose of this study, which addresses early COVID-19 diagnosis using CT images, is to improve the classification results (such as false-negative (FN), false-positive (FP) and total number of misclassified images (FN + FP)) by using new pipeline approaches. Additionally, this study aims to not increase in the classification time during this improvement.

Within the scope of this study, convolutional neural network (CNN), local binary pattern (LBP), and dual-tree complex wavelet transform (DT-CWT) methods were used to achieve this goal. There are some previous studies in which LBP and DT-CWT methods were combined with CNN. LBP and DT-CWT were used in the recognition of palmprint images in a study carried out by Hardalac et al. [44]. In that study, it was shown that using LBP as a pre-treatment improves its results. Zhang et al. [45] and Ke et al. [46] utilized these two methods for face recognition. In the study conducted by Yang et al. [47], LBP and CNN were combined in facial-expression recognition. Touahri et al. [48] used these two methods in breast cancer diagnosis. In these studies, LBP was generally used as a pre-treatment; LBP-based CNN was introduced by Juefei-Xu et al. [49].

In 2017, DT-CWT and CNN were used for gear fault diagnosis in a study by Sun et al. [50]. In that study, DT-CWT was used in a one-dimensional signal separation of gear audio signals. In 2018, Lu et al. [51] proposed an algorithm using these two methods for human thyroid medical image segmentation.

CNN, a deep learning method, was used in this study. COVID-19/non-COVID-19 classification results were obtained for the cases by directly using images, LBP as a pre-processing step, and DT-CWT as a secondary processing step; the results were combined using five different pipeline approaches. The same procedures were repeated for COVID-19 pneumonia/other pneumonia classification. The results show that by using these approaches, diagnostic success (COVID-19/non-COVID-19 and COVID-19 pneumonia/other pneumonia classification) can be improved by reducing the classification time cost. It is estimated that the proposed pipeline approaches will make an important contribution to the literature in general because they apply to the studies conducted by other researchers.

Methods

Used Data

Along with the emergence of COVID-19, many academic studies have been conducted in which CT lung and chest X-ray images are clinically evaluated. Some researchers have created datasets based on metadata, including the images published in these academic studies, and have made them available to the public. CT lung images of COVID-19 patients used in this study were obtained by Cohen et al. [52] and Zhao et al. [53] by merging the datasets based on metadata that were created in this context and made available to the public via GitHub. During the merging process, the images and their related clinical notes were considered. As a result, a merged COVID-19 image dataset containing 386 CT lung images was created. Images taken from the same patient in the dataset were taken on different days during the course of the disease and by different imaging shots. Real images of patients, confirmed by PCR testing and collected from the public Hospital of the Government Employees of Sao Paulo (HSPM) and the Metropolitan Hospital of Lapa (both in Sao Paulo, Brazil) by Soares et al. [54, 55], were also used in this study. In this context, the number of COVID-19 images used in this study increased to 2554 by including 2168 COVID-19 images.

An examination of the horizontal and vertical image sizes and their recording formats showed that their sizes varied between 115 × 98 and 2024 × 1523; their formats were png, jpg, and jpeg. The majority of images were 24-bit RGB, while a small portion was 8-bit greyscale. Since the visual features in question showed such differences, they had to be standardized for use in this study. First, COVID-19 CT lung images were converted to 8-bit greyscale; next, the images were framed within the lung region borders, and the study area of interest was defined. After these standardization processes, all the images were rearranged, adjusted to 448 × 448, and saved in png format.

Some non-COVID-19 CT images used in this study were taken from the LIDC-IDRI dataset [56,57,58], which was created for use in cancer research. It was completed in 2015 and subsequently made available for public access. The dataset contains CT images of normal lungs, that is, having no cancer nodules smaller or larger than 3 mm, as well as CT images of lungs that have cancer nodules. Therefore, the dataset is a merged dataset containing varied images of normal lungs and intensive cancer lungs. CT lung images of 1010 patients in the LIDC-IDRI dataset were used for this study. The CT lung images of all the patients in the dataset were used, and no elimination or exclusion was made. More detailed information about the dataset can be obtained from studies that published the dataset [56,57,58] and from the open-access extension [59] of the dataset. This study included 756 healthy CT lung images from the dataset published by Soares et al. [54, 55]. Therefore, the total number of non-COVID-19 images used in the study increased to 1766.

In addition to the COVID-19/non-COVID-19 classification, COVID-19 pneumonia/other pneumonia classification was conducted. The images used for the other pneumonia class were obtained from a database that was created and made available by Soares et al. [54, 55]. This study used 1247 images of other pneumonia.

The same processes applied to COVID-19 CT lung images were applied to non-COVID-19 and other pneumonia images, and these images were also standardized. In Fig. 1, COVID-19, non-COVID-19, and other pneumonia CT lung images are shown. The images’ summary information is given in Table 3. If a comparison between the total number of images used in the study and the number of images used in previous studies in the literature is made, the average number of images used in 24 studies for COVID-19/non-COVID-19 classification is 1896. In this study, the total number of images used in this classification area was 4320 (2254 COVID-19 and 1766 non-COVID-19). In this study, the number images used was more than 2 times the average, more images were used than 23 of the 24 previous studies, and almost an equal number as the 24th. For COVID-19 pneumonia/other pneumonia classification, an average of 1011 images were used in 7 studies in the literature, while the number of images used in this study was 3801 (2254 COVID-19 and 1247 other pneumonia). Approximately four times the number of images used in this study, more than 91% related to COVID-19/non-COVID-19 classification were taken directly from the real world, and 90% were taken for COVID-19 pneumonia/other pneumonia classification. The main purpose of increasing the number of images used was to minimize the overfitting problem that can be encountered in CNN training.

Fig. 1
figure 1

a COVID-19, b non-COVID-19, and c other pneumonia CT lung image samples

Full size image
Table 3 Information on CT lung images used in this study
Full size table

Local Binary Pattern

LBP is a simple yet highly effective tissue-extraction operator that produces results by comparing a centre pixel with its neighbours [60]. In the LBP operation, if the value of a neighbouring pixel is greater than or equal to the central pixel, the neighbouring pixel will be 1, and if it is smaller, it will be 0; therefore, binary values are assigned to the pixels around the centre pixel. The new value of the centre pixel is calculated by multiplying these binary values with the weight values that are calculated, depending on the order of the neighbouring pixels. As shown in Fig. 2, a sample LBP operation was carried out for a circle with a radius value of 1. It is possible to run LBP operations for many radius values, starting from 1. The radius values for LBP used in the first-stage experiments of this study were 1, 2, and 3. However, considering the length of the study, in later stage experiments, the process was continued using the radius value that provided the highest success in the first-stage experiments. Figure 3 shows sample images obtained by applying the LBP operation to CT lung images used for this study.

Fig. 2
figure 2

A sample LBP transaction

Full size image
Fig. 3
figure 3

Images obtained by applying LBP to some a COVID-19, b non-COVID-19, and c other pneumonia CT lung images in Fig. 1 (radius values from left to right are 1, 2, and 3, respectively)

Full size image

Dual-Tree Complex Wavelet Transform

Wavelet transform is a method of transitioning to a frequency plane that is widely used in image-processing applications. Applying a discrete wavelet filter to an image allows four sub-band image matrices called LL, LH, HL, and HH to be obtained. The dimensions of the sub-band image matrices are half the image dimensions. It is also possible to obtain lower sub-band image matrices for different scale values by using the LL sub-band image matrix obtained in the first stage.

DT-CWT, which is based on the use of two wavelet filters operating in parallel, was first proposed by Kingsbury [61, 62]. One of the DT-CWT filters allows the real part of the sub-band image matrix to be obtained, and the other obtained the imaginary part. In this way, it is possible to perform operations in a greater number of directions than with the standard discrete wavelet filter, thereby obtaining the lower-band image matrix. At the end of the DT-CWT process, operations are performed in six different directions: + 15, − 15, + 45, − 45, + 75, and − 75°. Figure 4 shows the sub-band images obtained when DT-CWT was applied to an image. To reveal the effect on the results of the change in the scale parameter, the process was continued by using two different scale values: 1 and 2.

Fig. 4
figure 4

DT-CWT decomposition scheme

Full size image

Convolutional Neural Network

Deep learning is a subfield of artificial intelligence that derives new data from existing data by using multiple hidden layers and any machine learning method. In practical applications, machine learning is mostly preferred over an artificial neural network. CNN is the most commonly used deep learning model, although there are other models, such as recurrent neural networks, deep belief networks, and autoencoders.

CNN consists of a convolution layer, an activation function, a pooling layer, flattening, and a fully connected layer. Descriptions for these components are briefly given below.

  • The convolution layer is the layer where the convolution processes are carried out. The process is performed by dividing the image into small pieces according to the size of the convolution filter to be used. In principle, the convolution layers are placed one after another and allow image-feature patterns to be extracted. This extraction starts with the low-level features to continue to the high-level features. Following the convolution process, a process of normalization is also carried out through a normalization inter-layer.

  • Activation functions generate new outputs from their inputs, according to their functions. These functions are among many alternatives that bring the inputs into a certain range, taking some of the inputs and dropping others, or changing the direction of the inputs.

  • The dimensions of the feature matrices, obtained as a result of the convolution processes, are excessive for direct use. For this reason, they should be reduced by sampling. Pooling layers are the layers where these sampling processes are carried out.

  • The image property matrices obtained by convolution, the activation function, and pooling are converted from a matrix to a vector before being transferred to the fully connected layer where the classification is made. This process is called flattening.

  • The fully connected layer is where the outputs, i.e. the classification results, are obtained by using flattened-feature vectors as inputs. This layer can, in essence, be described as a conventional neural network.

In all the experiments conducted for this study, a 24-layer CNN architecture was used. Table 4 shows the information and parameter properties of the CNN architecture layers. For Table 4, specifically adjusted parameters were noted, and other parameters were used as defaults. The experiments were carried out using the MATLAB 2019 (a) software, and detailed information about these specific and default parameters can be viewed on the MathWorks [63] page.

Table 4 Features of the CNN architecture used for this study
Full size table

Special attention was given to the CNN architecture to prevent the vanishing gradient problem, so ReLU (Rectified Linear Unit), which is a saturate-in-one-direction-only function, was preferred. Accordingly, a total of five ReLU layers were added to the architecture and situated behind the convolution layers. Additionally, batch normalization was performed by adding a batch normalization layer between convolution and ReLU layers, thus preventing the destruction of small derivatives. The CNN architecture used in this study included five batch normalization layers in total.

Another issue considered in CNN design is the overfitting problem. To avoid this problem, a dropout layer was added just before the fully connected layer in the CNN architecture. In this way, half the data (0.5) in the fully connected layer entry was randomly set to zero, and the network was prevented from memorizing. The purpose of keeping the dataset much larger than in the literature studies, as explained before, was to prevent overfitting. As an additional measure, the training set data were randomly shuffled before each iteration during network training.

The parameters used for the CNN training are given in Table 5 so that the experiments can be repeated by other study groups if required. Among these values, the defaults used by MATLAB 2019 (a) are indicated separately in parentheses, and comprehensive descriptions of the parameters can be viewed on the MathWorks [64] page. During network training, no data augmentation method or process was used.

Table 5 Information on the CNN training options
Full size table

No transfer was made for the starting weights during the CNN training. In other words, the training phase was run from the very beginning, and each experiment was repeated five times to limit the effects of the initial weights randomly assigned by the program to the results and thereby stabilize the study’s results. The average values of the experimental repetitions were shared within the scope of this study.

Evaluation Criteria of the Classification Results

The confusion matrix, one of the most basic evaluation methods, was used to evaluate and compare the results. The confusion matrix consists of four numerical components, TP, FP, TN, and FN, for a classification consisting of two groups. TP and TN are the numbers of patient data labelled patient and normal, respectively, and as a result of the classification, the actual statuses are patient and normal, respectively. FP is the number of normal data points labelled as patients as a result of the classification, and FN is the number of actual patient data points labelled as normal. Many comparison parameters, such as sensitivity, specificity, precision, negative predictive value, false discovery rate, false omission rate, accuracy, and F1 score, are calculated using confusion-matrix components (TP, FP, TN, and FN). Therefore, the confusion-matrix results of all the experiments were added specifically to the result tables. In this way, if other researches need to perform a comparison, the calculation of these parameters can be easily made, even if they are not given in this study.

In addition to the confusion-matrix components, sensitivity (SEN), specificity (SPE), accuracy (ACC), and F-1 score (F-1) values were calculated and included. Sensitivity measures the ratio of the true positives labelled correctly. Specificity determines the ratio of the true negatives labelled correctly. Accuracy is the ratio of the number of correctly labelled data points to the total number of data points. Mathematical calculations of these parameters were performed with the formulas in Eqs. (1)–(2).

$${\text{SEN}} = {{{\text{TP}}} \mathord{\left/ {\vphantom {{{\text{TP}}} {\left( {{\text{TP}} + {\text{FN}}} \right)}}} \right. \kern-\nulldelimiterspace} {\left( {{\text{TP}} + {\text{FN}}} \right)}}$$
(1)
$${\text{SPE}} = {{{\text{TN}}} \mathord{\left/ {\vphantom {{{\text{TN}}} {\left( {{\text{TN}} + {\text{FP}}} \right)}}} \right. \kern-\nulldelimiterspace} {\left( {{\text{TN}} + {\text{FP}}} \right)}}$$
(2)
$${\text{ACC}} = {{\left( {{\text{TP}} + {\text{TN}}} \right)} \mathord{\left/ {\vphantom {{\left( {{\text{TP}} + {\text{TN}}} \right)} {\left( {{\text{TP}} + {\text{TN}} + {\text{FP}} + {\text{FN}}} \right)}}} \right. \kern-\nulldelimiterspace} {\left( {{\text{TP}} + {\text{TN}} + {\text{FP}} + {\text{FN}}} \right)}}$$
(3)
$${\text{F - 1 score}} = {{\left( {2 \times {\text{TP}}} \right)} \mathord{\left/ {\vphantom {{\left( {2 \times {\text{TP}}} \right)} {\left( {2 \times {\text{TP}} + {\text{FP}} + {\text{FN}}} \right)}}} \right. \kern-\nulldelimiterspace} {\left( {2 \times {\text{TP}} + {\text{FP}} + {\text{FN}}} \right)}}$$
(4)

The threshold values used to label the results in determining the confusion-matrix components TP, FP, TN, and FN are of great importance. For example, in labelling a result set with classification results ranging from 0 to 1, different confusion matrices were obtained for the case when the threshold value was 0.5 and 0.9; the labels were made for the threshold value to be accepted as 0.5.

Another basic evaluation method, receiver operating characteristic (ROC) analysis, was used, and the area under the ROC curve (area under the curve (AUC)) was determined. The ROC curve is the change in sensitivity (SEN) (y-axis) relative to precision, namely, the complement of specificity to 1 (1-SPE) (x-axis) depending on the variation in the threshold value used for labelling between the minimum and maximum result values predicted for classification.

Proposed Pipeline Approach for COVID-19/Non-COVID-19 and COVID-19 Pneumonia/Other Pneumonia Classification

First, the results of the experiment were calculated for the case when the original CT lung images were used directly, and the images were obtained by applying LBP to these original CT lung images. Next, the results were obtained for the case when some combinations of sub-image matrices, obtained by using DT-CWT transformation as an intermediate process in classification processes, were used. In this way, classification results were obtained with and without application of the LBP process. Figure 5 shows a summary of the experiments carried out with this approach.

Fig. 5
figure 5

Summary of the first-stage experiments carried out in this study

Full size image

Second, pipeline approaches, based on combining the results obtained in the first stage, were proposed, and their success was tested. Five pipeline approaches were used, where the first, second, and third pipeline approaches were based on the creation of the new classification results made through the combination, in certain proportions, of the numerical results obtained in the context of the two experiments, which relate to the cases where the diagnosis of labels was performed with, and without, the help of the LBP process (COVID-19 or non-COVID-19 for COVID-19/non-COVID-19 classification; COVID-19 pneumonia or other pneumonia for COVID-19 pneumonia/other pneumonia classification). If these labels differed, new diagnosis labels were affiliated with these results. In Fig. 6, the approach related to the first three pipelines is explained visually. New numerical results were obtained for three different weight combinations, with the sum of the weight rates being 100%, 25%–75%, 50%–50%, and 75%–25%. Next, diagnostic labelling of the new numerical classification result was carried out. In these pipeline approaches, if the diagnostic labels obtained with and without the help of the LBP process were identical and if these labels were COVID-19, then the minimum numerical result was accepted; if these labels were non-COVID-19, then the maximum numerical result was accepted as the new classification result for COVID-19/non-COVID-19 classification.

Fig. 6
figure 6

Block diagram of the pipeline approaches (Pipeline-1, Pipeline-2, and Pipeline-3) proposed in this study

Full size image

Similarly, if the diagnostic labels obtained with and without the help of the LBP process were identical and if these labels were COVID-19 pneumonia, then the minimum numerical result was accepted; if these labels were other pneumonia, then the maximum numerical result was accepted as the new classification result for COVID-19 pneumonia/other pneumonia classification.

The main reason for this is that a value of 0 was assigned as the target for COVID-19 images and a 1 value was assigned as the target for non-COVID-19 images in the course of training and test processes for COVID-19/non-COVID-19 classification. Similarly, 0 was assigned as the target for COVID-19 pneumonia images, and 1 was assigned as the target for other pneumonia images for COVID-19 pneumonia/other pneumonia classification.

The fourth and fifth pipeline approaches are explained in Figs. 7 and 8. The essential difference in these approaches from the first three is that if the diagnosis labels with and without the help of the LBP process were different, then unidirectional new numerical result calculations were performed. Here, unidirectional means only the COVID-19-labelled or non-COVID-19-labelled images for COVID-19/non-COVID-19 classification and COVID-19 pneumonia-labelled or other pneumonia-labelled images for COVID-19 pneumonia/other pneumonia classification. In this pipeline approach, a 50%–50% weight combination was used in the combining process. The other new numerical results and label acquisition procedures were similar to those of the first three pipelines.

Fig. 7
figure 7

Block diagram of Pipeline-4 proposed in this study

Full size image
Fig. 8
figure 8

Block diagram of Pipeline-5 proposed in this study

Full size image

Experiments and Results

Experiments

This study used a deep learning method, one of the current artificial intelligence approaches, to automatically distinguish COVID-19/non-COVID-19 and COVID-19 pneumonia/other pneumonia from CT lung images. CT lung images of 2554 COVID-19 patients, obtained by combining three datasets, and 1766 non-COVID-19 CT lung images with normal to extremely dense level nodules, obtained by combining two datasets, were used for COVID-19/non-COVID-19 classification. Similarly, CT lung images of 2554 COVID-19 patients, obtained by combining three datasets, and 1247 other pneumonia CT lung images were used for COVID-19 pneumonia/other pneumonia classification. The images were first framed to determine the interests of CT lung images. While the framing was being done, action was taken to cover the borders of the lung region to the maximum extent. CNN, one of the deep learning methods, was used, and all images used in training and test procedures were the same size for it to work. After framing, the images were resized, and the image sizes were set to 448 × 448. In the last stage of the pre-processing section, the images were saved as 8-bit greyscale in png format. One thing to note about the image sizes within the scope of this study is that after LBP processing, the image dimensions were re-set to 448 × 448 because image sizes decreased due to the application of the LBP process. Furthermore, the scale values of the DT-CWT operation used in this study were 1 and 2, and each application of this operation reduced image sizes by half.

A 24-layer CNN architecture, previously described in detail, was used for the training and testing section of this study. This CNN architecture was used for all training and test processes. However, since image and feature matrices of different sizes were used, only the image sizes given to the CNN input differed.

For the experiments, the training procedures were carried out according to the k-fold cross-validation principle, where the crossing value (k) was determined to be 10.

In every stage of the training process, 3889/3879 images were used in the training processes, except for 431/441 images (nine groups consisted of 431 images and one group consisted of 441 images) that were used in the test processes for COVID-19/non-COVID-19 classification. These training and test processes were run ten times, and the results for the classification of all the images were obtained. While 0 was assigned as the target value for COVID-19 images, 1 was assigned for non-COVID-19 images. The COVID-19 and non-COVID-19 labels of the numerical results were obtained, and the threshold value was accepted as 0.5. Numerical image results below this threshold value were labelled COVID-19, and numerical results above this value were labelled non-COVID-19.

In every stage of the training process, 3422/3411 images were used for the training processes, except for 379/390 images (nine groups consisting of 379 images and one group consisting of 390 images) that were used in test processes for COVID-19 pneumonia/other pneumonia classification. These training and test processes were run ten times, and the results for classification of all the images were obtained. While 0 was assigned as the target value for COVID-19 pneumonia images, 1 was assigned for other pneumonia images, and the threshold value was accepted as 0.5. Numerical image results below this threshold value were labelled COVID-19 pneumonia, and numerical results above this value were labelled other pneumonia.

Experiments were conducted, and results were obtained for both the images to which LBP was applied and those to which LBP was not applied, as described above. Next, the pipeline approaches, which allowed the results of these experiments to be combined and were previously explained, were set to work. The success of the five pipeline approaches was tested. No transfer was made for the starting weights during the CNN training. In other words, the training phase was run from the very beginning. In this context, each experiment was repeated five times to limit the effects of the initial weights randomly assigned by the program on the results and thereby stabilize the study’s results. The average values of the experimental repetitions were shared within the scope of this study.

To compare the operating times of the tested pipeline approaches, the period of CPU times in which results for the images were obtained were also measured. The experiments were conducted with the help of the MATLAB 2019 (a) software running on an Intel(R) Xeon (R) CPU E5-2680 2.7-GHz (32-CPU) computer with 64 GB RAM.

Results

First, training and test processes were carried out (without using DT-CWT) using the images obtained by both applying and not applying the LBP process to greyscale CT lung images. In these experiments, the size of the image matrices given as input to the CNN architecture was 448 × 448 × 1. Next, the results of these two experiments were combined with the help of the five different pipeline approaches detailed in the previous sections, and new results were calculated. In these first-stage experiments, the effect of the change in radius value used in the LBP process on the results was also examined. For this reason, the results for the images on which LBP was applied for three different radius values, 1, 2, and 3, were shared. The results are given in Tables 6, 7, and 8 for COVID-19/non-COVID-19 classification and in Tables 9, 10, and 11 for COVID-19 pneumonia/other pneumonia classification. The highest results obtained in Tables 6, 7, 8, 9, 10, and 11 (the same as in other tables) are marked in bold. The tested pipeline approaches increased the study results for all the parameters used in the evaluation.

Table 6 Results obtained by directly using CT lung images for COVID-19/non-COVID-19 classification (LBP radius value 1)
Full size table
Table 7 Results obtained by directly using CT lung images for COVID-19/non-COVID-19 classification (LBP radius value 2)
Full size table
Table 8 Results obtained by directly using CT lung images for COVID-19/non-COVID-19 classification (LBP radius value 3)
Full size table
Table 9 Results obtained by directly using CT lung images for COVID-19 pneumonia/other pneumonia classification (LBP radius value 1)
Full size table
Table 10 Results obtained by directly using CT lung images for COVID-19 pneumonia/other pneumonia classification (LBP radius value 2)
Full size table
Table 11 Results obtained by directly using CT lung images for COVID-19 pneumonia/other pneumonia classification (LBP radius value 3)
Full size table

Second, training and testing processes were carried out, and the results were obtained by using the real part of the LL and LLL sub-bands obtained by applying DT-CWT as an intermediate process to the CT lung images to which LBP was applied and not applied. Since the highest results obtained in the first-stage experiments were obtained when the LBP radius value was 1, only the results of this radius value are given for the processes after this stage. In these experiments, the size of the image matrices given as an input to the CNN architecture was 224 × 224 × 1 for the LL sub-band and 112 × 112 × 1 for the LLL sub-band. The results obtained in this context are given in Tables 12 and 13 for COVID-19/non-COVID-19 classification and in Tables 14 and 15 for COVID-19 pneumonia/other pneumonia classification.

Table 12 Results obtained by using the real part of the LL sub-band obtained by applying DT-CWT to CT lung images for COVID-19/non-COVID-19 classification
Full size table
Table 13 Results obtained by using the real part of the LLL sub-band obtained by applying DT-CWT to CT lung images for COVID-19/non-COVID-19 classification
Full size table
Table 14 Results obtained by using the real part of the LL sub-band obtained by applying DT-CWT to CT lung images for COVID-19 pneumonia/other pneumonia classification
Full size table
Table 15 Results obtained by using the real part of the LLL sub-band obtained by applying DT-CWT to CT lung images for COVID-19 pneumonia/other pneumonia classification
Full size table

Third, training and testing processes were carried out, and the results were obtained by using the imaginary part of the LL and LLL sub-bands obtained by applying DT-CWT as an intermediate process to the CT lung images to which LBP was not applied. In these experiments, the sizes of the image matrices given as input to the CNN architecture were 224 × 224 × 1 for the LL sub-band and 112 × 112 × 1 for the LLL sub-band. The results are given in Tables 16 and 17 for COVID-19/non-COVID-19 classification and in Tables 18 and 19 for COVID-19 pneumonia/other pneumonia classification.

Table 16 Results obtained by using the imaginary part of the LL sub-band obtained by applying DT-CWT to CT lung images for COVID-19/non-COVID-19 classification
Full size table
Table 17 Results obtained by using the imaginary part of the LLL sub-band obtained by applying DT-CWT to CT lung images for COVID-19/non-COVID-19 classification
Full size table
Table 18 Results obtained by using the imaginary part of the LL sub-band obtained by applying DT-CWT to CT lung images for COVID-19 pneumonia/other pneumonia classification
Full size table
Table 19 Results obtained by using the imaginary part of the LLL sub-band obtained by applying DT-CWT to CT lung images for COVID-19 pneumonia/other pneumonia classification
Full size table

Fourth, training and testing processes were carried out, and the results were obtained by using the real part of the LL, LH, and HL sub-bands and the real part of the LLL, LLH, and LHL sub-bands obtained by applying DT-CWT as an intermediate process to CT lung images to which LBP was applied and not applied. In these experiments, the size of the image matrices given as an input to the CNN architecture was 224 × 224 × 3 for the LL, LH, and HL sub-bands and 112 × 112 × 3 for the LLL, LLH, and LHL sub-bands. The results are given in Tables 20 and 21 for COVID-19/non-COVID-19 classification and in Tables 22 and 23 for COVID-19 pneumonia/other pneumonia classification.

Table 20 Results obtained by using the real part of the LL, LH, and HL sub-bands obtained by applying DT-CWT to CT lung images for COVID-19/non-COVID-19 classification
Full size table
Table 21 Results obtained by using the real part of the LLL, LLH, and LHL sub-bands obtained by applying DT-CWT to CT lung images for COVID-19/non-COVID-19 classification
Full size table
Table 22 Results obtained by using the real part of the LL, LH, and HL sub-bands obtained by applying DT-CWT to CT lung images for COVID-19 pneumonia/other pneumonia classification
Full size table
Table 23 Results obtained by using the real part of the LLL, LLH, and LHL sub-bands obtained by applying DT-CWT to CT lung images for COVID-19 pneumonia/other pneumonia classification
Full size table

Fifth, training and testing processes were carried out, and the results were obtained by using the imaginary part of the LL, LH, and HL sub-bands and the imaginary part of the LLL, LLH, and LHL sub-bands obtained by applying DT-CWT as an intermediate process to CT lung images to which LBP was applied and not applied. In these experiments, the size of the image matrices given as an input to the CNN architecture was 224 × 224 × 3 for the LL, LH, and HL sub-bands and 112 × 112 × 3 for the LLL, LLH, and LHL sub-bands. The results are given in Tables 24 and 25 for COVID-19/non-COVID-19 classification and in Tables 26 and 27 for COVID-19 pneumonia/other pneumonia classification.

Table 24 Results obtained by using the imaginary part of the LL, LH, and HL sub-bands obtained by applying DT-CWT to CT lung images for COVID-19/non-COVID-19 classification
Full size table
Table 25 Results obtained by using the imaginary part of the LLL, LLH, and LHL sub-bands obtained by applying DT-CWT to CT lung images for COVID-19/non-COVID-19 classification
Full size table
Table 26 Results obtained by using the imaginary part of LL, LH, and HL sub-bands obtained by applying DT-CWT to CT lung images for COVID-19 pneumonia/other pneumonia classification
Full size table
Table 27 Results obtained by using the imaginary part of LLL, LLH, and LHL sub-bands obtained by applying DT-CWT to CT lung images for COVID-19 pneumonia/other pneumonia classification
Full size table

Sixth, training and testing processes were carried out, and the results were obtained by using the real and imaginary parts of the LL and LLL sub-bands obtained by applying DT-CWT as an intermediate process to CT lung images to which LBP was applied and not applied. In these experiments, the size of the image matrices given as an input to the CNN architecture was 224 × 224 × 2 for the LL sub-band and 112 × 112 × 2 for the LLL sub-band. The results are given in Tables 28 and 29 for COVID-19/non-COVID-19 classification and in Tables 30 and 31 for COVID-19 pneumonia/other pneumonia classification.

Table 28 Results obtained by using the real and imaginary parts of the LL sub-band obtained by applying DT-CWT to CT lung images for COVID-19/non-COVID-19 classification
Full size table
Table 29 Results obtained by using the real and imaginary parts of the LLL sub-band obtained by applying DT-CWT to CT lung images for COVID-19/non-COVID-19 classification
Full size table
Table 30 Results obtained by using the real and imaginary parts of the LL sub-band obtained by applying DT-CWT to CT lung images for COVID-19 pneumonia/other pneumonia classification
Full size table
Table 31 Results obtained by using the real and imaginary parts of the LLL sub-band obtained by applying DT-CWT to CT lung images for COVID-19 pneumonia/other pneumonia classification
Full size table

Conclusion

The results of the experiments for COVID-19/non-COVID-19 and COVID-19 pneumonia/other pneumonia classifications are shared in Tables 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, and 31. In this section, the results are evaluated.

It was seen that the application of LBP as a pre-process to the images had some effects on the study results, and using the images related to COVID-19/non-COVID-19 classification without applying LBP as a pre-process provided better results in general than when LBP was applied as a pre-process; however, there were some exceptions. Exceptions occurred for the specificity parameter if DT-CWT was implemented as a secondary process. In addition, the increase in the radius values used in the LBP process negatively affected the results. For this reason, considering the length of the study, only the results for the case where the radius value was 1 in the experiments performed after the first stage are shared. The summary information of the test results obtained by applying, and not applying, LBP as pre-processing for COVID-19/non-COVID-19 classification is given in Table 32.

Table 32 Summary information of the study results obtained with and without LBP for COVID-19/non-COVID-19 classification
Full size table

When the effect of applying LBP on the images related to COVID-19 pneumonia/other pneumonia classification was examined, it was seen that there was a different situation compared with COVID-19/non-COVID-19 classification. In the experiments performed without DT-CWT as a secondary process, the test results for the case when the LBP radius value was 1, obtained by applying LBP as a pre-process, were better than the results of the experiments where LBP pre-processing was not applied. However, if the radius value used in the LBP process increased or DT-CWT was used as the secondary process, the test results obtained by applying LBP as a pre-process fell behind the results obtained without applying LBP. The decrease in the results, due to the increase in the radius value used in the LBP process, was also valid for COVID-19 pneumonia/other pneumonia classification. The summary information of the test results obtained by applying and not applying LBP as a pre-process for COVID-19 pneumonia/other pneumonia classification is included in Table 33.

Table 33 Summary information of the study results obtained with and without LBP for COVID-19 pneumonia/other pneumonia classification
Full size table

The effect of using DT-CWT as a secondary procedure on the results is another topic that needs to be studied. The summary information of the results obtained by using DT-CWT (scale values 1 and 2) and not using DT-CWT for COVID-19/non-COVID-19 classification is given in Table 34. As seen in Table 34, if DT-CWT was used as a secondary process, there was a certain amount of change in the results. While these changes were limited for the scale value 1, the amount of these changes increased depending on the increase in the scale value. However, if a comparison is made in terms of the per-image speed of producing results, there was a significant reduction in processing times due to the use of DT-CWT. The summary information for COVID-19 pneumonia/other pneumonia classification is included in Table 35, which shows that there is a similar situation for this classification heading.

Table 34 Summary information of the study results obtained with and without DT-CWT for COVID-19/non-COVID-19 classification
Full size table
Table 35 Summary information of the study results obtained with and without DT-CWT for COVID-19 pneumonia/other pneumonia classification
Full size table

Significant improvements in the results were achieved by using the recommended pipeline approaches. Table 36 provides information on the best FN, FP, FN + FP, and 1-AUC values obtained for the experiments carried out before and after using the pipeline approaches for COVID-19/non-COVID-19 classification and shows the rates of change related to these values. Table 37 shows, as does Table 36, the aforementioned information for COVID-19 pneumonia/other pneumonia classification.

Table 36 Information on the best FN, FP, FN + FP and 1-AUC values obtained for the experiments before and after using the pipeline approaches for COVID-19/non-COVID-19 classification
Full size table
Table 37 Information on the best FN, FP, FN + FP and 1-AUC values obtained for the experiments before and after using the pipeline approaches for COVID-19 pneumonia/other pneumonia classification
Full size table

As seen from Table 36, with the aid of pipeline approaches, improvements were achieved for COVID-19/non-COVID-19 classification at rates varying between 42.7 and 54.9% in the FN parameter, 40.9 and 53.8% in the FP parameter, 17.1 and 29.3% in the FN + FP parameter, and 20.9 and 41.1% in the 1-AUC parameter. Similarly, as shown in Table 37, improvements were achieved for COVID-19 pneumonia/other pneumonia classification at rates varying between 65.1 and 78% in the FN parameter, 27.4 and 33.5% in the FP parameter, 13.9 and 21.9% in the FN + FP parameter, and 23.5 and 41.2% in the 1-AUC parameter.

The highest sensitivity, specificity, accuracy, F-1, and AUC values obtained without using the pipeline approach for COVID-19/non-COVID-19 classification were 0.9676, 0.9181, 0.9456, 0.9545, and 0.9890, respectively. The highest sensitivity, specificity, accuracy, F-1, and AUC values obtained by using the pipeline approach in the same classification heading were 0.9832, 0.9622, 0.9577, 0.9642, and 0.9923, respectively. The comparison of these results, which showed the highest scores achieved, with previous studies is shown in Table 38. The highest sensitivity value obtained through pipeline approaches was achieved using the real and imaginary parts of the LL sub-band as input data with the help of the Pipeline-4 approach. The highest specificity and accuracy values were achieved using the imaginary part of the LL sub-band as input data with the help of the Pipeline-5 approach. The highest F-1 and AUC values were achieved using the same input data with the help of the Pipeline-2 approach.

Table 38 Comparison of the results with previous studies for COVID-19/non-COVID-19 classification
Full size table

The highest sensitivity, specificity, accuracy, F-1, and AUC values obtained for COVID-19 pneumonia/other pneumonia classification without using the pipeline approach were 0.9615, 0.7270, 0.8846, 0.9180, and 0.9370, respectively. The highest sensitivity, specificity, accuracy, F-1, and AUC values obtained by using the pipeline approach in the same classification heading were 0.9915, 0.8140, 0.9071, 0.9327, and 0.9615, respectively. The comparison of these results, which showed the highest scores achieved, with previous studies is shown in Table 39. The highest sensitivity value obtained through pipeline approaches was achieved using the real part of the LL, LH, and HL sub-bands as input data with the help of the Pipeline-4 approach. The highest specificity and accuracy values were achieved using the same input data with the help of the Pipeline-5 approach. The highest F-1 and AUC values were achieved using the same input data with the help of the Pipeline-2 approach.

Table 39 Comparison of the results with previous studies for COVID-19 pneumonia/other pneumonia classification
Full size table

Discussion

This study makes suggestions for new pipeline approaches that were tested to reduce the number of FN, FP, and total misclassified images (FN + FP) obtained from CT lung images in COVID-19 diagnosis, and important results were obtained. First, COVID-19/non-COVID-19 classification test results were obtained with and without the help of LBP processes, and then new classification results were calculated by combining pipeline approaches with these results. The classifications related to the first two experiments were carried out through one of the actual deep learning methods, that is, CNN. Moreover, DT-CWT, as an intermediate process, was put to use to reduce the size of the image in some experimental groups.

If a comparison of the classification results obtained with and without the help of the LBP process was made, it could be said that, in general, the results obtained were more successful if LBP was not used; only in some experimental groups did LBP provide better results. In this respect, it can be seen that using LBP as a pre-process alone does not increase but decreases the classification accuracy. It was also seen that the periods of time for producing results related to the experiments conducted with and without the help of LBP were close when the results per image were compared. From the experiments, it is understood that the LBP process produces the best results if 1 is selected as a radius.

The results were calculated using sub-band matrix and matrix combinations obtained by using DT-CWT as a secondary process in addition to the experiments conducted with and without the help of LBP. In general, the DT-CWT process kept the general features of the image intact, thus reducing its dimensions. However, as the scale value used in the DT-CWT process increased, the change in the results also increased. Using the pipeline approach generally doubled the processing load. Choosing 1 for the scale value in the DT-CWT process was sufficient to tolerate this increase in processing load. In this context, it is estimated that using a scale value of 1 in the DT-CWT process is the most appropriate choice.

The results obtained with and without the help of LBP were combined through pipeline approaches, and new results were calculated. The results show that in all experimental groups, comparison parameters can be increased using pipeline approaches. This shows that the highest sensitivity values in the experimental groups were obtained by using the fourth pipeline approach, the highest specificity values were obtained by using the fifth pipeline approach, and the highest accuracy for F-1 and AUC values in the experimental groups were obtained by using the second and fifth pipeline approaches.

Because the pipeline approaches’ operating time was less than one-thousandth of a second per image, the operating times of the pipeline approaches were approximately equal to the total operating times of the first two experiments. This approximate equation is valid for all experimental groups. The classification time per image was 1.1915s (for COVID-19/non-COVID-19 classification) and 1.1733s (for COVID-19 pneumonia/other pneumonia classification) without operating the pipeline and without applying LBP or DT-CWT processes to the images. If the method of combining the results with the pipeline approaches and the use of the proposed application of LBP and DT-CWT to the images were combined, the classification time for the case with the highest time cost was 0.6488 and 0.6409 s, respectively. This means that the classification approach to produce the results per image proposed in this study reduced the time by at least half, and it also provided a significant improvement in the classification results.

The results support different studies [44,45,46,47,48,49] where the use of CNN and LBP methods applied together and the use of LBP increased the success of the relevant studies. However, in those studies, the use of LBP appears to be a factor that directly improves the results. The use of LBP in this study did not directly increase the result, and it often decreased it. However, if used with the proposed pipeline algorithm, the results significantly improved. In this respect, it differed from other studies. In addition, this is the first study in which the LBP, DT-CWT, and CNN methods were used together, and it makes an important contribution to the literature.

When the results for COVID-19/non-COVID-19 classification are compared with the results obtained in previous studies (information of which is given in Tables 1 and 38), it can be seen that significant results were obtained in this study. In this context, it is understood that better results have been obtained than two-thirds of the studies carried out in the literature. For COVID-19 pneumonia/other pneumonia classification, higher results were obtained from six of the seven studies performed in the literature. Since the number of images used in this study was considerably higher than in other studies, it is estimated that it is more valid by comparison.

In the studies to be carried out after this stage, the aim is to make an automatic classification of chest X-ray and ultrasound images, which are important diagnostic tools similar to CT lung images in the classification of COVID-19, with the aid of the new pipeline approaches used in this study and those to be developed after. It is also planned to combine the results using direct transfer learning by using pipeline approaches. The use of the analysis of the effects using multi-resolution analyses other than DT-CWT on the study results is also seen as an important subject of study. In contrast, it is estimated that another important application will be the classification of complex-valued sub-image bands obtained through multi-resolution analysis by directly using a complex-valued CNN.