Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for second-trimester anomaly screening, for which ultrasound (US) is employed. Although expert sonographers are adept at reading US images, MR images which closely resemble anatomical images are much easier for non-experts to interpret. Thus in this article we propose to generate MR-like images directly from clinical US images. In medical image analysis such a capability is potentially useful as well, for instance for automatic US-MRI registration and fusion. The proposed model is end-to-end trainable and self-supervised without any external annotations. Specifically, based on an assumption that the US and MRI data share a similar anatomical latent space, we first utilise a network to extract the shared latent features, which are then used for MRI synthesis. Since paired data is unavailable for our study (and rare in practice), pixel-level constraints are infeasible to apply. We instead propose to enforce the distributions to be statistically indistinguishable, by adversarial learning in both the image domain and feature space. To regularise the anatomical structures between US and MRI during synthesis, we further propose an adversarial structural constraint. A new cross-modal attention technique is proposed to utilise non-local spatial information, by encouraging multi-modal knowledge fusion and propagation. We extend the approach to consider the case where 3D auxiliary information (e.g., 3D neighbours and a 3D location index) from volumetric data is also available, and show that this improves image synthesis. The proposed approach is evaluated quantitatively and qualitatively with comparison to real fetal MR images and other approaches to synthesis, demonstrating its feasibility of synthesising realistic MR images.

中文翻译：

---

用于MRI胎儿脑图像合成的自我监督超声。

胎儿脑磁共振成像（MRI）可提供发育中的大脑的精美图像，但不适用于采用超声（US）的孕中期异常筛查。尽管专业的超声检查专家善于阅读美国图像，但是与非解剖学图像非常相似的MR图像对于非专业人士来说更容易理解。因此，在本文中，我们建议直接从临床US图像生成MR图像。在医学图像分析中，这种功能也可能有用，例如对于US-MRI自动配准和融合。所提出的模型是端到端可训练的，并且是自我监督的，没有任何外部注释。具体来说，基于US和MRI数据共享相似的解剖潜伏空间的假设，我们首先利用网络提取共享的潜伏特征，然后将其用于MRI合成。由于配对数据无法用于我们的研究（在实践中很少见），因此像素级约束不适用。相反，我们建议通过在图像域和特征空间中进行对抗性学习，使分布在统计上无法区分。为了使合成过程中US和MRI之间的解剖结构规范化，我们进一步提出了对抗性结构约束。提出了一种新的跨模式注意技术，通过鼓励多模式知识的融合和传播来利用非局部空间信息。我们扩展该方法以考虑来自体积数据的3D辅助信息（例如3D邻居和3D位置索引）也可用的情况，并证明这可以改善图像合成。