B-mode ultrasonography and sonoelastography are used in the clinical diagnosis of prostate cancer (PCa). A combination of the two ultrasound (US) modalities using computer aid may be helpful for improving the diagnostic performance. A technique for computer-aided diagnosis (CAD) of PCa is presented based on multimodal US. Firstly, quantitative features are extracted from both B-mode US images and sonoelastograms, including intensity statistics, regional percentile features, gray-level co-occurrence matrix (GLCM) texture features and binary texture features. Secondly, a deep network named PGBM-RBM2 is proposed to learn and fuse multimodal features, which is composed of the point-wise gated Boltzmann machine (PGBM) and two layers of the restricted Boltzmann machines (RBMs). Finally, the support vector machine (SVM) is used for prostatic disease classification. Experimental evaluation was conducted on 313 multimodal US images of the prostate from 103 patients with prostatic diseases (47 malignant and 56 benign). Under five-fold cross-validation, the classification sensitivity, specificity, accuracy, Youden’s index and area under the receiver operating characteristic (ROC) curve with the PGBM-RBM2 were 87.0%, 88.8%, 87.9%, 75.8% and 0.851, respectively. The results demonstrate that multimodal feature learning and fusion using the PGBM-RBM2 can assist in the diagnosis of PCa. This deep network is expected to be useful in the clinical diagnosis of PCa.

中文翻译：

---

用于前列腺癌诊断的点状门控深度网络在 B 模式超声和声弹性成像上的多模态特征学习和融合

B 型超声和超声弹性成像用于前列腺癌 (PCa) 的临床诊断。使用计算机辅助的两种超声 (US) 模式的组合可能有助于提高诊断性能。提出了一种基于多模态超声的 PCa 计算机辅助诊断 (CAD) 技术。首先，从 B 模式超声图像和超声弹性图中提取定量特征，包括强度统计、区域百分位数特征、灰度共生矩阵 (GLCM) 纹理特征和二值纹理特征。其次，一个名为 PGBM-RBM 的深度网络2提出了学习和融合多模态特征，它由逐点门控玻尔兹曼机（PGBM）和两层受限玻尔兹曼机（RBM）组成。最后，支持向量机（SVM）用于前列腺疾病分类。对来自 103 名前列腺疾病患者（47 名恶性和 56 名良性）的 313 幅多模态美国前列腺图像进行了实验评估。五折交叉验证下，PGBM-RBM的分类灵敏度、特异性、准确度、约登指数和ROC曲线下面积2分别为 87.0%、88.8%、87.9%、75.8% 和 0.851。结果表明，使用 PGBM-RBM 进行多模态特征学习和融合2可辅助 PCa 的诊断。这种深度网络有望在 PCa 的临床诊断中发挥作用。