Since the human brain functional mechanism has been enabled for investigation by the functional magnetic resonance imaging (fMRI) technology, simultaneous modeling of both the spatial and temporal patterns of brain functional networks from 4-D fMRI data has been a fundamental but still challenging research topic for neuroimaging and medical image analysis fields. Currently, general linear model (GLM), independent component analysis (ICA), sparse dictionary learning, and recently deep learning models, are major methods for fMRI data analysis in either spatial or temporal domains, but there are few joint spatial–temporal methods proposed, as far as we know. As a result, the 4-D nature of fMRI data has not been effectively investigated due to this methodological gap. The recent success of deep learning applications for functional brain decoding and encoding greatly inspired us in this paper to propose a novel framework called spatio–temporal convolutional neural network (ST-CNN) to extract both spatial and temporal characteristics from targeted networks jointly and automatically identify of functional networks. The identification of default mode network (DMN) from fMRI data was used for evaluation of the proposed framework. Results show that only training the framework on one fMRI data set is sufficiently generalizable to identify the DMN from different data sets of different cognitive tasks and resting state. Further investigation of the results shows that the joint-learning scheme can capture the intrinsic relationship between the spatial and temporal characteristics of DMN and thus it ensures the accurate identification of DMN from independent data sets. The ST-CNN model brings new tools and insights for fMRI analysis in cognitive and clinical neuroscience studies.

中文翻译：

---

通过时空卷积神经网络 (ST-CNN) 对 fMRI 数据进行 4D 建模

由于功能性磁共振成像 (fMRI) 技术可以研究人脑功能机制，因此从 4-D fMRI 数据同时建模大脑功能网络的空间和时间模式一直是一个基本但仍然具有挑战性的研究课题用于神经影像和医学图像分析领域。目前，一般线性模型（GLM）、独立分量分析（ICA）、稀疏字典学习和最近的深度学习模型是空间或时间域 fMRI 数据分析的主要方法，但很少提出联合时空方法， 据我们了解。因此，由于这种方法上的差距，fMRI 数据的 4-D 性质尚未得到有效研究。最近在功能性大脑解码和编码方面的深度学习应用的成功极大地启发了我们在本文中提出一种称为时空卷积神经网络 (ST-CNN) 的新框架，以联合从目标网络中提取时空特征并自动识别的功能网络。从 fMRI 数据中识别默认模式网络 (DMN) 用于评估所提出的框架。结果表明，仅在一个 fMRI 数据集上训练框架就足以从不同认知任务和静息状态的不同数据集中识别 DMN。对结果的进一步调查表明，联合学习方案可以捕捉 DMN 的空间和时间特征之间的内在关系，从而确保从独立数据集中准确识别 DMN。ST-CNN 模型为认知和临床神经科学研究中的 fMRI 分析带来了新的工具和见解。