Multimodal registration is a challenging task due to the significant variations exhibited from images of different modalities. CT and MRI are two of the most commonly used medical images in clinical diagnosis, since MRI with multicontrast images, together with CT, can provide complementary auxiliary information. The deformable image registration between MRI and CT is essential to analyze the relationships among different modality images. Here, we proposed an indirect multimodal image registration method, i.e., sCT-guided multimodal image registration and problematic image completion method. In addition, we also designed a deep learning-based generative network, Conditional Auto-Encoder Generative Adversarial Network, called CAE-GAN, combining the idea of VAE and GAN under a conditional process to tackle the problem of synthetic CT (sCT) synthesis. Our main contributions in this work can be summarized into three aspects: (1) We designed a new generative network called CAE-GAN, which incorporates the advantages of two popular image synthesis methods, i.e., VAE and GAN, and produced high-quality synthetic images with limited training data. (2) We utilized the sCT generated from multicontrast MRI as an intermediary to transform multimodal MRI-CT registration into monomodal sCT-CT registration, which greatly reduces the registration difficulty. (3) Using normal CT as guidance and reference, we repaired the abnormal MRI while registering the MRI to the normal CT.

中文翻译：

---

一种以图像合成为指导的间接多峰图像配准和完成方法。

由于从不同模态的图像显示出很大的变化，因此多模态配准是一项具有挑战性的任务。CT和MRI是临床诊断中最常用的两种医学图像，因为具有多对比度图像的MRI与CT可以提供互补的辅助信息。MRI和CT之间的可变形图像配准对于分析不同形态图像之间的关系至关重要。在这里，我们提出了一种间接的多峰图像配准方法，即sCT引导的多峰图像配准和有问题的图像完成方法。此外，我们还设计了一个基于深度学习的生成网络，称为CAE-GAN的条件自动编码器生成对抗网络，在有条件的过程中结合了VAE和GAN的思想，以解决合成CT（sCT）合成问题。我们在这项工作中的主要贡献可以归纳为三个方面：（1）设计了一种新的生成网络，称为CAE-GAN，它结合了两种流行的图像合成方法（VAE和GAN）的优势，并生成了高质量的合成图像。训练数据有限的图像。（2）我们利用多对比度MRI产生的sCT作为中介，将多峰MRI-CT配准转换为单峰sCT-CT配准，大大降低了注册难度。（3）以正常CT为指导和参考，在将MRI记录到正常CT的同时修复了异常MRI。并以有限的训练数据生成了高质量的合成图像。（2）我们利用多对比度MRI产生的sCT作为中介，将多峰MRI-CT配准转换为单峰sCT-CT配准，大大降低了注册难度。（3）以正常CT为指导和参考，在将MRI记录到正常CT的同时修复了异常MRI。并以有限的训练数据生成了高质量的合成图像。（2）我们利用多对比度MRI产生的sCT作为中介，将多峰MRI-CT配准转换为单峰sCT-CT配准，大大降低了注册难度。（3）以正常CT为指导和参考，在将MRI记录到正常CT的同时修复了异常MRI。