Comparison of machine learning approaches for enhancing Alzheimer’s disease classification

The overall classification scheme

Our AD classification consisted of three major steps: (i) data acquisition and pre-processing including data transformation, bias filed correction, skull-stripping, and registration; (ii) feature selection for support vector machine; (iii) deep learning algorithms with feature selection automatically. Here, two different approaches, 3D-VGGNet and 3D-ResNet, were used in the deep learning module (Fig. 1). A total of 560 MR images composed of 260 AD and 300 cognitive normal were used in the study.

Data acquisition and preprocess

We obtained 560 T1-weighted MR images from Alzheimers Disease Neuroimaging Initiative (ADNI). The dataset included 260 AD and 300 cognitive normal (CN) older adults. All the images are in 3D (170 × 256 × 256) and acquired from 81 ADNI participants, including 42 female and 39 male subjects with an average age of 75 years old (Table S1).

The raw data of structural MRI scans for both the AD and the CN groups were provided in DICOM format in the ADNI database. The initial step was to convert DICOM format to NIfTI, as analysis on NIfTI images is more convenient than on DICOM. After that, each individual NIfTI images was corrected to uniform intensity based on the N4 bias field correction algorithm (Tustison et al., 2010). All non-brain tissues were removed from the data. Then, we registered the images to the MNI template (MNI152 T1 one mm brain) (Grabner, Neubauer & Stern, 2006) by a 12-parameter affine transformation using the FLSR package. Image registration aligned all images into one coordinate system, which is necessary for comparing or integrating the data sets. After registration to the MNI template, the dimension of the 3D volume was 182 × 218 × 182.

SVM based model

Each MRI volume was treated as a feature vector in a high-dimensional space, where each feature was the MRI signal at a specific voxel. We applied the SVM algorithm to map whole MRI volumes from different subjects to brain states without prior selection of features. In an MRI scan, an image volume often contains thousands of voxels, resulting in a very large spatial dimension of MRI data. Training an SVM classifier for feature vectors with such high dimensions is quite computationally intensive. Therefore, data reduction is important. A mask generated from the raw images was used to remove the out of brain voxels to enhance the performance of the classifier. Dimension was reduced by selecting 10% relevant voxels using ANOVA—an F-score based feature selection (https://nilearn.github.io/decoding/decoding_intro.html). The linear kernel was adopted in the SVM classifier. Cross-validation was used for performance evaluation.

Network architecture

The 3D-VGGNet (Simonyan & Zisserman, 2014) we used contained four blocks of 3D convolutional layers and 3D max pooling layers, followed by a fully connected layer, a batch normalization layer, a dropout layer, another fully connected layer, and the softmax output layer to produce the probabilities of AD and CN. More details of the 3D-VGGNet model architecture were presented in Fig. S1. Batch sizes of 5 were assigned to 3D-VGGNet to cope with the limited GPU memory. The deep networks were stable after iteration of about 50 epochs. Adam (Dauphin, De Vries & Bengio, 2015) with a 2.7e⁻⁵ learning rate was used as the optimizing function. The small learning rate was used here to avoid overshooting the minimum. The categorical cross-entropy was adopted as the model loss function.

The 3D-ResNet model was composed of six residual blocks (Fig. S2). Each residual block consisted of two 3D convolutional layers with 3 × 3 × 3 filters that have a batch normalization layer and a rectified- linear-unit nonlinearity layer (ReLU) between them. Each residual block fed the output of two convolutional layers and one ReLU layers and also bypassed the input to the following layers. Skip connections (identity mapping of the input) added a residual block element-by-element to the following residual block, explicitly allowing the following block to learn a residual mapping instead of a full mapping. For optimization, Nesterovs accelerated stochastic gradient descent (Nesterov, 2003) was used. Optimization parameters were set as a 2.7e⁻⁵ for learning rate with Adam optimizer, four for batch size, and 80 for training epochs. The same loss function as 3D-VGGNet and the two-class cross-entropy function were employed. The final fully connected layer of 3D-ResNet yielded the probabilities of AD and CN with softmax transformation. More details of the 3D-ResNet model architecture were presented in Fig. S2.

Transfer learning

We applied a transfer learning strategy to prevent overfitting problem in the training of the deep learning models. The weights of a pre-trained model were transferred and adopted as the initial weights of our models. The previously trained model was chosen from (Yang, Rangarajan & Ranka, 2018). We leveraged knowledge from the pre-trained source model with the features learned from our training data to construct the AD classification model. A small learning rate was adopted for reducing the risk of losing previous knowledge. Transferring the learned features significantly reduced training time and increased network efficiency.

Mapping disease regions

We used a Grad-CAM method (Selvaraju et al., 2017), highlighting brain parts that were essential for Alzheimers disease classification, to obtain visual explanations. Grad-CAM utilizes gradient information of the target class, which flows into the last convolutional layer and contains the spatial information indicating discriminative regions for classifications. The results showed the coarse locations of image discriminative features that were used by the model for classification.

Environment set up for deep learning algorithms

The scripts for implementing the deep learning algorithm were written in Python 3.5. All software and library dependencies bundled were packaged in a Singularity container (Kurtzer, Sochat & Bauer, 2017), ensuring quick and easy reproducibility of the results. The two neural networks were implemented in the Lasagne library with Theano (Bastien et al., 2012) as the GPU backend support. The experiments were conducted on a 12GB Nvidia Tesla GPU card.

The performance of SVM based model

We applied the SVM algorithm to map whole MRI volumes from different subjects to brain states without prior selection of features. As each image contains many thousands of voxels, to reduce the computational cost, a mask generated from the raw images was applied to remove the voxels outside of the brain. Then, the features obtained from the ANOVA feature selector were used as inputs to a linear kernel SVM classifier. We used 500 images for training and the remaining 60 for testing. The model achieved around 90% classification accuracy with a sensitivity of 93.94% and specificity of 85.19%. We employed stratified 5-fold validation to evaluate the robustness of the classifier. The resulting mean value of AUC (area under curve) was 0.97 (Fig. 2).

The SVM classifier returned a weight vector, representing the importance of individual features for separating distinct cognitive states. As we used the reduced representation of data as input to the SVM classifier, we essentially obtained the weight vector for the reduced representation. To recover the volume with the most discriminating regions in the original space, we mapped the weight vector back to the high-dimensional space. As a result, we obtained a discriminative map, showing the brain regions that contributed most to AD classification (Fig. 3).

The performance of deep learning-based models

The MR images were split into training and testing datasets, containing 500 and 60 images, respectively. Then, through random shuffling, 80% of training data were assigned for actual training and the remaining images for validation. The validation set was used for model optimization in the training process. The testing dataset was the same for 3D-VGGNet and ResNet models, and it was independent of training and validation sets. The voxel-wise features of one MRI sample were of high dimensionality while our sample size was limited. Hence, there were far more features than the training subjects, which can lead to an overfitting problem and poor performance. We employed techniques including batch normalization, dropout, and cross-validation in our deep network to prevent overfitting.

We first construct the classification models from scratch by randomly initializing model parameters. During the training of the 3D-VGGNet model, we found that the training loss was decreasing whereas the validation loss increased over time (Fig. S3), suggesting the overfitting. The 3D-ResNet model was failed to build from the scratch as the required memory exceeded our GPU memory capacity even after the batch size was adjusted to two (Table S2).

Thus, we applied a transfer learning strategy for model construction. The weights of a pre-trained model were transferred and adopted (Yang, Rangarajan & Ranka, 2018) as the initial weights of our models. Compared to the model trained from scratch, we found that utilizing transfer learning increased classification accuracy while reduced the training convergence time. The validation accuracy and AUC of 3D-VGGNet were increased from 0.759 to 0.97 and 0.74 to 0.994, respectively (Table S2). The training time per epoch reduced from 788s to 123s (Table S2). Collectively, our results suggested that transferring learning significantly reduced the training time and boosted efficiency of the network. The models that were trained using transfer learning were utilized in our classification and the subsequent analysis.

The transfer learning-based modes achieved high accuracy without the overfitting problem. Both training loss and validation loss decreased over the training epochs (Figs. 4A, 4B). In the training process, the performance 3D-VGGNet and 3D-ResNet classifiers constantly improved and then plateaued after the 50th epoch and 80th epoch, respectively (Figs. 4C–4F). We applied these two models to the 60 testing images and both classifiers achieved 95% accuracy and similar AUC values, 0.994 vs 0.995 (Table 1). Compared to 3D-ResNet model, the 3D-VGGnet based model yielded higher specificity (100% vs 96%), whereas a lower sensitivity (91.4% vs 94.3%)

Discriminative regions visualization

We further combined the CNN models with the Grad-CAM (Selvaraju et al., 2017) approach to visualize important areas that contributed to the classification. The last convolutional layer of the CNN contains the spatial information indicating discriminative regions for making classifications. The white matter region was highlighted in normal samples (Figs. 5A–5C) as compared to AD images. Convergent neuroimaging studies have implicated micro- and macro-structural abnormalities in white matter associated with the risk and progression of AD. White matter alterations have been reported to be associated with a decline in motor function in speed and fine motor coordination (Bozzali et al., 2002; Kao et al., 2019; Nasrabady et al., 2018).

On the other hand, the cerebral cortex (the outer layer of the brain) area was highlighted (Figs. 5D–5F) in the AD image as well as in the SVM discriminative map sporadically (Fig. 3), indicating the cerebral cortex might relate to Alzheimers disease. Pathologically, Alzheimers disease is associated with neuronal death and gliosis specifically in the cerebral cortex (Li et al., 2018). Anatomical properties, such as volume, gray matter density, and thickness of the cerebral cortex have been used to identify AD in various ROI-based MRI quantitative analyses. Those studies have shown that thickness was reduced in regions of the cerebral cortex that were affected pathologically in AD (Bakkour, Morris & Dickerson, 2009; Dickerson et al., 2009; Lerch et al., 2005).

Interestingly, the cerebellum was highlighted in the maps generated based on 3D-VGGNet model (Figs. 5G– 5I) and in the one based on 3D-ResNet model (Fig. 5B). The cerebellum communicates with the cerebral cortex and is responsible for emotion and cognition, however, the cerebellum is a relatively neglected area of the AD brain and is not commonly be considered involved in the pathophysiology of Alzheimers disease (Jacobs et al., 2018). The spatial information we learned from our models suggested that the cerebellum may have roles in the clinical phenomenology of the disease.

Classification model comparison

A traditional machine learning algorithm and two deep 3D convolutional neural networks for Alzheimers disease classification were investigated in this study. Among them, the SVM classifier had the lowest accuracy (90%), and two CNN classifiers achieved the same accuracy of 95% on our testing dataset (Table 1). VGGNet is a common feed-forward network with convolutional and pooling layers, while ResNet is a modern residual neural network. The skip connections can facilitate the learning process for deeper networks in the ResNet model. Our 3D-ResNet model yielded higher sensitivity and AUC value than the 3D-VGGNet (Table 1) classifier.

We also compared the heatmaps generated by Grad-CAM based on the convolutional layer activation of the VGGNet and ResNet models. In the discriminative maps, the brain regions that were essential for classification were highlighted. We found that Res-3D-Grad-CAM generated the maps with higher resolution and accuracy than VGG-3D-Grad-CAM (Figs. 6D, 6E, 6F vs 6A, 6B, 6C). The Grad-CAM approach has a low-resolution problem and it produces a coarse heatmap of the same size as the last convolutional layer. The size of the last convolutional layers of our 3D-VGGNet and 3D-ResNet was 3 × 3 × 3 and 14 × 14 × 14, respectively. The smaller size of the convolutional layer of 3D-VGGNet model may result in loss of details and lower localization accuracy.