An increased interest in longitudinal neurodevelopment during the first few years after birth has emerged in recent years. Noninvasive magnetic resonance imaging (MRI) can provide crucial information about the development of brain structures in the early months of life. Despite the success of MRI collections and analysis for adults, it remains a challenge for researchers to collect high-quality multimodal MRIs from developing infant brains because of their irregular sleep pattern, limited attention, inability to follow instructions to stay still during scanning. In addition, there are limited analytic approaches available. These challenges often lead to a significant reduction of usable MRI scans and pose a problem for modeling neurodevelopmental trajectories. Researchers have explored solving this problem by synthesizing realistic MRIs to replace corrupted ones. Among synthesis methods, the convolutional neural network-based (CNN-based) generative adversarial networks (GANs) have demonstrated promising performance. In this study, we introduced a novel 3D MRI synthesis framework– pyramid transformer network (PTNet3D)– which relies on attention mechanisms through transformer and performer layers. We conducted extensive experiments on high-resolution Developing Human Connectome Project (dHCP) and longitudinal Baby Connectome Project (BCP) datasets. Compared with CNN-based GANs, PTNet3D consistently shows superior synthesis accuracy and superior generalization on two independent, large-scale infant brain MRI datasets. Notably, we demonstrate that PTNet3D synthesized more realistic scans than CNN-based models when the input is from multi-age subjects. Potential applications of PTNet3D include synthesizing corrupted or missing images. By replacing corrupted scans with synthesized ones, we observed significant improvement in infant whole brain segmentation.

中文翻译：

---

PTNet3D：基于 Transformer 的 3D 高分辨率纵向婴儿脑 MRI 合成器

近年来，人们对出生后最初几年对纵向神经发育的兴趣日益浓厚。无创磁共振成像 (MRI) 可以提供有关生命最初几个月大脑结构发育的重要信息。尽管成人 MRI 收集和分析取得了成功，但对于研究人员来说，从发育中的婴儿大脑中收集高质量的多模态 MRI 仍然是一个挑战，因为婴儿的睡眠模式不规则、注意力有限、无法按照指示在扫描过程中保持静止。此外，可用的分析方法有限。这些挑战通常会导致可用 MRI 扫描的显着减少，并为神经发育轨迹建模带来问题。研究人员已经探索通过合成真实的 MRI 来替换损坏的 MRI 来解决这个问题。在合成方法中，基于卷积神经网络 (CNN-based) 的生成对抗网络 (GAN) 已展示出良好的性能。在这项研究中，我们介绍了一种新颖的 3D MRI 合成框架——金字塔变换器网络（PTNet3D）——它通过变换器和执行器层依赖于注意力机制。我们对高分辨率开发人类连接组项目 (dHCP) 和纵向婴儿连接组项目 (BCP) 数据集进行了广泛的实验。与基于 CNN 的 GAN 相比，PTNet3D 在两个独立的大规模婴儿大脑 MRI 数据集上始终显示出卓越的合成精度和卓越的泛化能力。尤其，我们证明，当输入来自多年龄受试者时，PTNet3D 合成的扫描比基于 CNN 的模型更真实。PTNet3D 的潜在应用包括合成损坏或丢失的图像。通过用合成的扫描替换损坏的扫描，我们观察到婴儿全脑分割的显着改善。