Magnetic resonance imaging (MRI) is being increasingly utilized to assess, diagnose, and plan treatment for a variety of diseases. The ability to visualize tissue in varied contrasts in the form of MR pulse sequences in a single scan provides valuable insights to physicians, as well as enabling automated systems performing downstream analysis. However many issues like prohibitive scan time, image corruption, different acquisition protocols, or allergies to certain contrast materials may hinder the process of acquiring multiple sequences for a patient. This poses challenges to both physicians and automated systems since complementary information provided by the missing sequences is lost. In this paper, we propose a variant of generative adversarial network (GAN) capable of leveraging redundant information contained within multiple available sequences in order to generate one or more missing sequences for a patient scan. The proposed network is designed as a multi-input, multi-output network which combines information from all the available pulse sequences and synthesizes the missing ones in a single forward pass. We demonstrate and validate our method on two brain MRI datasets each with four sequences, and show the applicability of the proposed method in simultaneously synthesizing all missing sequences in any possible scenario where either one, two, or three of the four sequences may be missing. We compare our approach with competing unimodal and multi-modal methods, and show that we outperform both quantitatively and qualitatively.

中文翻译：

---

使用多模态生成对抗网络缺少MRI脉冲序列综合。

磁共振成像（MRI）越来越多地用于评估，诊断和计划各种疾病的治疗。在单次扫描中以MR脉冲序列的形式以各种对比显示组织的能力为医生提供了宝贵的见识，并且使自动化系统能够执行下游分析。但是，许多问题，例如扫描时间过长，图像损坏，不同的采集方案或对某些对比材料的过敏，可能会阻碍为患者采集多个序列的过程。由于丢失的序列所提供的补充信息丢失了，这对医生和自动化系统都构成了挑战。在本文中，我们提出了一种生成对抗网络（GAN）的变体，该变体能够利用多个可用序列中包含的冗余信息来生成一个或多个缺失序列供患者扫描。所提出的网络被设计为多输入，多输出网络，该网络将来自所有可用脉冲序列的信息组合在一起，并在单个前向通过中合成丢失的脉冲序列。我们在两个具有四个序列的大脑MRI数据集上论证并验证了我们的方法，并显示了该方法在同时缺失四个序列中的一个，两个或三个的任何可能情况下同时合成所有缺失序列的适用性。我们将我们的方法与竞争性单峰方法和多峰方法进行了比较，结果表明我们在数量和质量上均优于竞争对手。