PURPOSE The detection of clinically significant prostate cancer (PCa) is shown to greatly benefit from MRI-ultrasound fusion biopsy, which involves overlaying pre-biopsy MRI volumes (or targets) with real-time ultrasound images. In previous literature, machine learning models trained on either MRI or ultrasound data have been proposed to improve biopsy guidance and PCa detection. However, quantitative fusion of information from MRI and ultrasound has not been explored in depth in a large study. This paper investigates information fusion approaches between MRI and ultrasound to improve targeting of PCa foci in biopsies. METHODS We build models of fully convolutional networks (FCN) using data from a newly proposed ultrasound modality, temporal enhanced ultrasound (TeUS), and apparent diffusion coefficient (ADC) from 107 patients with 145 biopsy cores. The architecture of our models is based on U-Net and U-Net with attention gates. Models are built using joint training through intermediate and late fusion of the data. We also build models with data from each modality, separately, to use as baseline. The performance is evaluated based on the area under the curve (AUC) for predicting clinically significant PCa. RESULTS Using our proposed deep learning framework and intermediate fusion, integration of TeUS and ADC outperforms the individual modalities for cancer detection. We achieve an AUC of 0.76 for detection of all PCa foci, and 0.89 for PCa with larger foci. Results indicate a shared representation between multiple modalities outperforms the average unimodal predictions. CONCLUSION We demonstrate the significant potential of multimodality integration of information from MRI and TeUS to improve PCa detection, which is essential for accurate targeting of cancer foci during biopsy. By using FCNs as the architecture of choice, we are able to predict the presence of clinically significant PCa in entire imaging planes immediately, without the need for region-based analysis. This reduces the overall computational time and enables future intra-operative deployment of this technology.

中文翻译：

---

通过 MRI 和时间增强超声的信息融合改善前列腺癌病灶的检测。


目的 MRI-超声融合活检显示临床上显着的前列腺癌 (PCa) 的检测大大受益，其中涉及将活检前 MRI 体积（或目标）与实时超声图像叠加。在之前的文献中，已经提出了基于 MRI 或超声数据训练的机器学习模型来改进活检引导和 PCa 检测。然而，大规模研究尚未深入探讨 MRI 和超声信息的定量融合。本文研究了 MRI 和超声之间的信息融合方法，以改善活检中 PCa 病灶的定位。方法 我们使用来自新提出的超声模式、时间增强超声 (TeUS) 和来自 107 名患者（145 个活检核心）的表观扩散系数 (ADC) 的数据构建了全卷积网络 (FCN) 模型。我们模型的架构基于 U-Net 和带有注意力门的 U-Net。通过中间和后期数据融合，使用联合训练来构建模型。我们还分别使用每种模式的数据构建模型，以用作基线。根据预测临床意义 PCa 的曲线下面积 (AUC) 评估性能。结果 使用我们提出的深度学习框架和中间融合，TeUS 和 ADC 的集成优于癌症检测的单独模式。我们检测所有 PCa 病灶的 AUC 为 0.76，检测具有较大病灶的 PCa 的 AUC 为 0.89。结果表明，多种模态之间的共享表示优于平均单模态预测。 结论 我们证明了 MRI 和 TeUS 信息的多模态整合在改善 PCa 检测方面的巨大潜力，这对于活检过程中准确定位癌症病灶至关重要。通过使用 FCN 作为选择的架构，我们能够立即预测整个成像平面中是否存在具有临床意义的 PCa，而无需进行基于区域的分析。这减少了总体计算时间，并使该技术能够在未来的术中部署。