Purpose

Sparsity of annotated data is a major limitation in medical image processing tasks such as registration. Registered multimodal image data are essential for the diagnosis of medical conditions and the success of interventional medical procedures. To overcome the shortage of data, we present a method that allows the generation of annotated multimodal 4D datasets.

Methods

We use a CycleGAN network architecture to generate multimodal synthetic data from the 4D extended cardiac–torso (XCAT) phantom and real patient data. Organ masks are provided by the XCAT phantom; therefore, the generated dataset can serve as ground truth for image segmentation and registration. Realistic simulation of respiration and heartbeat is possible within the XCAT framework. To underline the usability as a registration ground truth, a proof of principle registration is performed.

Results

Compared to real patient data, the synthetic data showed good agreement regarding the image voxel intensity distribution and the noise characteristics. The generated T1-weighted magnetic resonance imaging, computed tomography (CT), and cone beam CT images are inherently co-registered. Thus, the synthetic dataset allowed us to optimize registration parameters of a multimodal non-rigid registration, utilizing liver organ masks for evaluation.

Conclusion

Our proposed framework provides not only annotated but also multimodal synthetic data which can serve as a ground truth for various tasks in medical imaging processing. We demonstrated the applicability of synthetic data for the development of multimodal medical image registration algorithms.

中文翻译：

---

为腹部医学图像配准生成带注释的多模态地面实况数据集

目的

注释数据的稀疏性是医学图像处理任务（例如配准）的主要限制。注册的多模态图像数据对于医疗状况的诊断和介入医疗程序的成功至关重要。为了克服数据短缺的问题，我们提出了一种允许生成带注释的多模态 4D 数据集的方法。

方法

我们使用 CycleGAN 网络架构从 4D 扩展心脏躯干 (XCAT) 模型和真实患者数据生成多模态合成数据。器官面具由 XCAT 幻影提供；因此，生成的数据集可以作为图像分割和配准的基本事实。在 XCAT 框架内可以对呼吸和心跳进行逼真的模拟。为了强调作为注册基础事实的可用性，执行了原则注册证明。

结果

与真实患者数据相比，合成数据在图像体素强度分布和噪声特征方面表现出良好的一致性。生成的T 1 加权磁共振成像、计算机断层扫描 (CT) 和锥形束 CT 图像本质上是共同配准的。因此，合成数据集允许我们优化多模态非刚性配准的配准参数，利用肝器官掩膜进行评估。

结论

我们提出的框架不仅提供了带注释的合成数据，而且还提供了多模态合成数据，这些数据可以作为医学成像处理中各种任务的基本事实。我们展示了合成数据在开发多模态医学图像配准算法中的适用性。