Hepatocellular Carcinoma (HCC) detection, size grading, and quantification (i.e. the center point coordinates, max-diameter, and area) by using multi-modality magnetic resonance imaging (MRI) are clinically significant tasks for HCC assessment and treatment. However, delivering the three tasks simultaneously is extremely challenging due to: (1) the lack of effective an mechanism to capture the relevance among multi-modality MRI information for multi-modality feature fusion and selection; (2) the lack of effective mechanism and constraint strategy to achieve mutual promotion of multi-task. In this paper, we proposed a task relevance driven adversarial learning framework (TrdAL) for simultaneous HCC detection, size grading, and multi-index quantification using multi-modality MRI (i.e. in-phase, out-phase, T2FS, and DWI). The TrdAL first obtains expressive feature of dimension reduction via using a CNN-based encoder. Secondly, the proposed modality-aware Transformer is utilized for multi-modality MRI features fusion and selection, which solves the challenge of multi-modality information diversity via capturing the relevance among multi-modality MRI. Then, the innovative task relevance driven and radiomics guided discriminator (Trd-Rg-D) is used for united adversarial learning. The Trd-Rg-D captures the internal high-order relationships to refine the performance of multi-task simultaneously. Moreover, adding the radiomics feature as the prior knowledge into Trd-Rg-D enhances the detailed feature extraction. Lastly, a novel task interaction loss function is used for constraining the TrdAL, which enforces the higher-order consistency among multi-task labels to enhance mutual promotion. The TrdAL is validated on a corresponding multi-modality MRI of 135 subjects. The experiments demonstrate that TrdAL achieves high accuracy of (1) HCC detection: specificity of 93.71%, sensitivity of 93.15%, accuracy of 93.33%, and IoU of 82.93%; (2) size grading: accuracy of large size, medium size, small size, tiny size, and healthy subject are 90.38%, 87.74%, 80.68%, 77.78%, and 96.87%; (3) multi-index quantification: the mean absolute error of center point, max-diameter, and area are 2.74mm, 3.17mm, and 144.51mm${}^2$. All of these results indicate that the proposed TrdAL provides an efficient, accurate, and reliable tool for HCC diagnosis in clinical.

使用多模态磁共振成像 (MRI) 对肝细胞癌 (HCC) 进行检测、大小分级和量化（即中心点坐标、最大直径和面积）是 HCC 评估和治疗的临床重要任务。然而，由于以下原因，同时执行这三个任务极具挑战性：（1）缺乏有效的机制来捕获多模态 MRI 信息之间的相关性以进行多模态特征融合和选择；(2)缺乏实现多任务相互促进的有效机制和约束策略。在本文中，我们提出了一个任务相关性驱动的对抗性学习框架 (TrdAL)，用于使用多模态 MRI（即同相、异相、T2FS 和 DWI）同时进行 HCC 检测、大小分级和多指标量化。TrdAL 首先通过使用基于 CNN 的编码器获得降维的表达特征。其次，所提出的模态感知 Transformer 用于多模态 MRI 特征融合和选择，通过捕获多模态 MRI 之间的相关性解决了多模态信息多样性的挑战。然后，创新的任务相关性驱动和放射组学引导鉴别器 (Trd-Rg-D) 用于联合对抗学习。Trd-Rg-D 捕获内部高阶关系以同时改进多任务的性能。此外，将放射组学特征作为先验知识添加到 Trd-Rg-D 中可以增强详细特征提取。最后，一种新的任务交互损失函数用于约束 TrdAL，它强制多任务标签之间的高阶一致性以增强相互促进。TrdAL 在 135 名受试者的相应多模态 MRI 上得到验证。实验表明，TrdAL 实现了（1）HCC 检测的高准确度：特异性为 93.71%，灵敏度为 93.15%，准确度为 93.33%，IoU 为 82.93%；(2)尺寸分级：大号、中号、小号、极小号、健康人的准确率分别为90.38%、87.74%、80.68%、77.78%、96.87%；(3)多指标量化：中心点、最大直径、面积的平均绝对误差分别为2.74mm、3.17mm、144.51mm 准确率为 93.33%，IoU 为 82.93%；(2)尺寸分级：大号、中号、小号、极小号、健康人的准确率分别为90.38%、87.74%、80.68%、77.78%、96.87%；(3)多指标量化：中心点、最大直径、面积的平均绝对误差分别为2.74mm、3.17mm、144.51mm 准确率为 93.33%，IoU 为 82.93%；(2)尺寸分级：大号、中号、小号、极小号、健康人的准确率分别为90.38%、87.74%、80.68%、77.78%、96.87%；(3)多指标量化：中心点、最大直径、面积的平均绝对误差分别为2.74mm、3.17mm、144.51mm${}^{\mathrm{2个}}$. 所有这些结果表明，所提出的 TrdAL 为临床 HCC 诊断提供了一种高效、准确和可靠的工具。