A diagnostic unified classification model for classifying multi-sized and multi-modal brain graphs using graph alignment

Highlights

• We propose a unified brain graph classification model that can classify multi-modal multisized brain graphs.

• We design a graph alignment strategy to a fixed graph template.

• Our unified classification model is diagnostic while being agnostic to the connectomic data source and size.

• Our proposed method is scalable and generalizable to different diagnostic clinical frameworks.

Abstract

Background

Presence of multimodal brain graphs derived from different neuroimaging modalities is inarguably one of the most critical challenges in building unified classification models that can be trained and tested on any brain graph regardless of its size and the modality it was derived from.

Existing methods

One solution is to learn a model for each modality independently, which is cumbersome and becomes more time-consuming as the number of modalities increases. Another traditional solution is to build a model inputting multimodal brain graphs for the target prediction task; however, this is only applicable to datasets where all samples have joint neuro-modalities.

New method

In this paper, we propose to build a unified brain graph classification model trained on unpaired multimodal brain graphs, which can classify any brain graph of any size. This is enabled by incorporating a graph alignment step where all multi-modal graphs of different sizes and heterogeneous distributions are mapped to a common template graph. Next, we design a graph alignment strategy to the target fixed-size template and further apply linear discriminant analysis (LDA) to the aligned graphs as a supervised dimensionality reduction technique for the target classification task.

Results

We tested our method on unpaired autistic and healthy brain connectomes derived from functional and morphological MRI datasets (two modalities).

Conclusion

Our results showed that our unified model method not only has great promise in solving such a challenging problem but achieves comparable performance to models trained on each modality independently.

Introduction

Multimodal magnetic resonance imaging (MRI) has introduced new exciting opportunities for understanding the brain as a complex system of interacting units in both health and disease. Based on MRI data, the brain can be represented as a connectome (i.e., graph), where the connection between different anatomical regions of interest (ROIs) is modeled. A large body of research work showed how the brain connectome gets altered by neurological and neuropsychiatric disorders, such as dementia and autism spectrum disorder (ASD) (Fornito et al., 2015, Van den Heuvel and Sporns, 2019). In network neuroscience (Bassett and Sporns, 2017), the brain connectome, which is typically regarded as a graph encoding the relationships between pairs of ROIs, presents a macroscale representation of the interactions between anatomical regions at functional, structural or morphological levels, derived from different multimodal MRI (e.g., functional resting-state MRI or T1-weighted MRI) (Bassett and Sporns, 2017). A bulk of connectome analysis studies used single modality to design unimodal machine learning frameworks for examining and classifying unimodal brain connectomes (Bi et al., 2018, Iidaka, 2014, Chen et al., 2011a, Chen et al., 2011b, Du et al., 2018, Cheng and Liu, 2017, Andersen et al., 2015, Zhang et al., 2015). For instance, a recent Kaggle competition paper (Bilgen et al., 2020) investigated the potential of cortical morphological networks in distinguishing between autistic and healthy subjects using different machine learning frameworks. Notably, single modality brain connectivity presents a reductionist view of the complexity of the brain construct, whereas multi-modal connectomic data provides complementary and more integral representations of the brain, which can be leveraged for designing learners trained on paired samples (i.e., each sample should have the same number of connectomic modalities) (Mesrob et al., 2012, Calhoun and Sui, 2016).

Although multi-modal brain connectomes present a rich dataset to design computer-aided diagnosis models; however, with their variation in resolution (i.e., connectome size or number of brain ROIs), heterogeneity, and lack of parity of samples across modalities, they can present several barriers to building learning-based classification models. In fact, it is not possible to train a conventional classification model (e.g., support vector machines) on such incoherent multimodal datasets in terms of the number of feature across samples. As an alternative solution, one can train a unimodal classifier on each modality, independently. However, this overlooks the complementarity and richness of multimodal connectomes and becomes more time-consuming as the number of modalities increases. Another solution is to combine different multimodal connectomes in the given population by simple concatenation and then train a single classifier using the concatenated feature vectors (Dai and He, 2014, Schouten et al., 2016, Raeper et al., 2018, Graa and Rekik, 2019, Corps and Rekik, 2019). However, such approach requires paired samples across modalities, i.e., all MRI modalities should be available for each subject in the population. This wastefully discards existing incomplete multi-modal MRI data as well as independent datasets collected from different sources with varying numbers of samples and available modalities.

To address these limitations, we propose a unified multimodal classification (UMC) model.¹ for classifying multi-sized and multi-modal brain graphs using graph alignment (Fig. 1). By aligning brain graphs from different modalities to a common template graph, we aim not only to eliminate the size problem we encounter in training a single model, but also avert to learn multiple classification models for multi-modal brain graphs. Our method does not stipulate sample parity across modalities since the multi-modal data are collectively mapped to fixed-size graphs via a shared template regardless of the number of available multimodal brain graphs (i.e., each subject can have one or more modalities).

Drawing inspiration from the recent work (Bai et al., 2019) on transforming arbitrary-sized graphs drawn from a single modality (i.e., source) into fixed-sized aligned grid structures for training deep learning models, we further extend it to handle multi-source heterogeneous connectomic datasets using four main steps: (i) extraction of feature vector embeddings using two different strategies: depth-based representation of each node in all brain graphs based on their local neighborhoods (Bai et al., 2019) and graph-wave node embeddings via diffusion wavelets (Donnat et al., 2018), (ii) clustering feature vectors into n_t clusters, each of which represents a node in the template graph to estimate, (iii) alignment of each graph from all modalities to the target template graph, and (iv) training a linear classifier using the fixed-sized aligned brain graphs. To the best of our knowledge, this study is the first to use a unified model for brain disorder classification on multi-modal data using depth-based alignment and presents several contributions. First, the proposed method can use any brain graph of any size derived from any neuroimaging modality in brain disease classification. Second, we do not need to build multiple classifiers for multi-modal data, but a unified classifier using aligned graphs. Third, multi-modal data do not have to be paired. Each available modality may include different number of brain networks collected from different cohorts. Fourth, our model can also handle data imbalance problem as the small sample size of a class (e.g., disordered) in one modality can be compensated with the large sample size of the same class in another modality. Our key contributions can be summarized as follows:

• Methodological. We pioneered a unified classification model for classifying multi-modal brain graphs with varying sizes using depth-based alignment to a unified connectional template, which eliminates the need to build multiple learners for different modalities and does not require paired graphs across modalities.

• Clinical. This develops the field of network neuroscience along the joint multi-source multi-modal data analysis front aiming to present a unified model for classifying disorders.

Remark 1

We set out to solve a challenging classification problem comprising multiple sub-problems. Solving them independently will certainly have a higher classification accuracy; however, our goal is to build a unified diagnostic framework that can be used with any modality of any size at any time point within the clinical routine. Simply put, our unified classification model is diagnostic while being agnostic to the connectomic data source and size, which makes it scalable and generalizable to different diagnostic clinical frameworks.

Remark 2

Our primary goal is to build a unified diagnostic framework that can be used with any modality of any size at any time point within the clinical routine, which might not allow to track the original brain connectivities between regions of interest, and thereby hinders the identification of potential biomarkers (i.e., disordered brain connections). However, in this paper, we prioritize model generalizability, adaptation and diagnosis accuracy over model interpretability and explainability.