The expanding adoption of digital pathology has enabled the curation of large repositories of histology whole slide images (WSIs), which contain a wealth of information. Similar pathology image search offers the opportunity to comb through large historical repositories of gigapixel WSIs to identify cases with similar morphological features and can be particularly useful for diagnosing rare diseases, identifying similar cases for predicting prognosis, treatment outcomes, and potential clinical trial success. A critical challenge in developing a WSI search and retrieval system is scalability, which is uniquely challenging given the need to search a growing number of slides that each can consist of billions of pixels and are several gigabytes in size. Such systems are typically slow and retrieval speed often scales with the size of the repository they search through, making their clinical adoption tedious and are not feasible for repositories that are constantly growing. Here we present Fast Image Search for Histopathology (FISH), a histology image search pipeline that is infinitely scalable and achieves constant search speed that is independent of the image database size while being interpretable and without requiring detailed annotations. FISH uses self-supervised deep learning to encode meaningful representations from WSIs and a Van Emde Boas tree for fast search, followed by an uncertainty-based ranking algorithm to retrieve similar WSIs. We evaluated FISH on multiple tasks and datasets with over 22,000 patient cases spanning 56 disease subtypes. We additionally demonstrate that FISH can be used to assist with the diagnosis of rare cancer types where sufficient cases may not be available to train traditional supervised deep models. FISH is available as an easy-to-use, open-source software package (https://github.com/mahmoodlab/FISH).

中文翻译：

---

快速且可扩展的组织学图像搜索

数字病理学的广泛采用使得能够管理包含大量信息的组织学全幻灯片图像 (WSI) 的大型存储库。相似的病理图像搜索提供了梳理千兆像素 WSI 的大型历史存储库以识别具有相似形态特征的病例的机会，并且对于诊断罕见疾病、识别相似病例以预测预后、治疗结果和潜在的临床试验成功特别有用。开发 WSI 搜索和检索系统的一个关键挑战是可扩展性，鉴于需要搜索越来越多的幻灯片，每个幻灯片都可以由数十亿像素组成，大小为几 GB，这具有独特的挑战性。这样的系统通常很慢，而且检索速度通常会随着它们搜索的存储库的大小而扩展，这使得它们的临床采用变得乏味，并且对于不断增长的存储库来说是不可行的。在这里，我们介绍了组织病理学快速图像搜索 (FISH)，这是一种组织学图像搜索管道，可无限扩展并实现恒定搜索速度，独立于图像数据库大小，同时具有可解释性且无需详细注释。FISH 使用自监督深度学习对来自 WSI 的有意义的表示进行编码，并使用 Van Emde Boas 树进行快速搜索，然后使用基于不确定性的排名算法来检索类似的 WSI。我们在多个任务和数据集上评估了 FISH，其中包含超过 22,000 个患者病例，涵盖 56 种疾病亚型。我们还证明了 FISH 可用于协助诊断罕见的癌症类型，其中可能没有足够的病例来训练传统的监督深度模型。FISH 作为易于使用的开源软件包 (https://github.com/mahmoodlab/FISH) 提供。