Content-based retrieval supports a radiologist decision making process by presenting the doctor the most similar cases from the database containing both historical diagnosis and further disease development history. We present a deep learning system that transforms a 3D image of a pulmonary nodule from a CT scan into a low-dimensional embedding vector. We demonstrate that such a vector representation preserves semantic information about the nodule and offers a viable approach for content-based image retrieval (CBIR). We discuss the theoretical limitations of the available datasets and overcome them by applying transfer learning of the state-of-the-art lung nodule detection model. We evaluate the system using the LIDC-IDRI dataset of thoracic CT scans. We devise a similarity score and show that it can be utilized to measure similarity 1) between annotations of the same nodule by different radiologists and 2) between the query nodule and the top four CBIR results. A comparison between doctors and algorithm scores suggests that the benefit provided by the system to the radiologist end-user is comparable to obtaining a second radiologist's opinion.

中文翻译：

---

Nodule2vec：使用语义表示进行肺结节检索的 3D 深度学习系统

基于内容的检索通过向医生展示包含历史诊断和进一步疾病发展历史的数据库中最相似的病例来支持放射科医生的决策过程。我们提出了一个深度学习系统，该系统将来自 CT 扫描的肺结节的 3D 图像转换为低维嵌入向量。我们证明了这种向量表示保留了有关结节的语义信息，并为基于内容的图像检索 (CBIR) 提供了一种可行的方法。我们讨论了可用数据集的理论局限性，并通过应用最先进的肺结节检测模型的迁移学习来克服它们。我们使用胸部 CT 扫描的 LIDC-IDRI 数据集评估系统。我们设计了一个相似度分数，并表明它可以用来衡量 1) 不同放射科医生对同一结节的注释之间的相似性，以及 2) 查询结节与前四个 CBIR 结果之间的相似性。医生和算法分数之间的比较表明，系统为放射科医师最终用户提供的好处与获得第二位放射科医师的意见相当。