As a common neurological disorder in early childhood and adolescence, an efficient and accurate diagnosis of Attention-Defect/Hyperactivity Disorder (ADHD) has always been one of the important goals in the field of psychiatry. However, most of the current diagnostic methods are based on a single table or fMRI images, which may result in the loss of much complementary information useful for diagnosis. Therefore, this paper presents a strategy for multimodal data fusion and an ensemble learning model (Trans3D-ensemble) to classify the ADHDs and typicals. The base classifiers of Trans3D-ensemble include two parts, Trans3D (Transformer for 3D images) to extract spatio-temporal features from fMRI images and random forest to extract clinical features from phenotypic data. Specifically, Trans3D utilizes 3D-CNN to capture volumetric spatial information and convert the 3D images into patch embeddings. Meanwhile, temporal pooling operation fuses the images tokens output by Transformer encoders across time and obtains the representative features of fMRI samples. Existing methods basically only build the above independent models, and few consider the effective combination of the those for diagnosis and treatment. Looking into this issue, we use stacking, one of ensemble learning methods, on multimodal data by merging the outputs from the two base classifiers. The proposed method reaches excellent results compared to most methods based on single modality and gets the accuracy of 74.5% on ADHD-200 data set, demonstrating the potential of the combination of phenotypic data and fMRI for ADHD diagnosis. Our strategy of multimodal data fusion provides a novel comprehensive diagnosis mode and the above results show that the proposed Trans3D-ensemble can effectively improve the auxiliary diagnosis of ADHD.

作为儿童早期和青春期常见的神经系统疾病，注意力缺陷/多动障碍的有效和准确诊断（ADHD）一直是精神病学领域的重要目标之一。然而，目前的大多数诊断方法都是基于单个表格或 fMRI 图像，这可能会导致许多对诊断有用的补充信息丢失。因此，本文提出了一种多模态数据融合策略和一个集成学习模型（Trans3D-ensemble）来对 ADHD 和典型进行分类。Trans3D-ensemble 的基本分类器包括两部分，Trans3D（Transformer for 3D images）从 fMRI 图像中提取时空特征和随机森林，从表型数据中提取临床特征。具体来说，Trans3D 利用 3D-CNN 捕获体积空间信息并将 3D 图像转换为补丁嵌入。同时，时间池化操作融合了 Transformer 编码器跨时间输出的图像标记，并获得 fMRI 样本的代表性特征。现有的方法基本上只建立了上述独立的模型，很少考虑将这些模型进行有效组合进行诊断和治疗。研究这个问题，我们通过合并来自两个基本分类器的输出，对多模态数据使用堆叠（一种集成学习方法）。与大多数基于单一模态的方法相比，所提出的方法取得了优异的结果，在 ADHD-200 数据集上获得了 74.5% 的准确率，证明了表型数据和 fMRI 相结合用于 ADHD 诊断的潜力。