Deep learning (DL) based semantic segmentation methods have achieved excellent performance in biomedical image segmentation, producing high quality probability maps to allow extraction of rich instance information to facilitate good instance segmentation. While numerous efforts were put into developing new DL semantic segmentation models, less attention was paid to a key issue of how to effectively explore their probability maps to attain the best possible instance segmentation. We observe that probability maps by DL semantic segmentation models can be used to generate many possible instance candidates, and accurate instance segmentation can be achieved by selecting from them a set of “optimized” candidates as output instances. Further, the generated instance candidates form a well-behaved hierarchical structure (a forest), which allows selecting instances in an optimized manner. Hence, we propose a novel framework, called hierarchical earth mover’s distance (H-EMD), for instance segmentation in biomedical 2D+time videos and 3D images, which judiciously incorporates consistent instance selection with semantic-segmentation-generated probability maps. H-EMD contains two main stages: (1) instance candidate generation: capturing instance-structured information in probability maps by generating many instance candidates in a forest structure; (2) instance candidate selection: selecting instances from the candidate set for final instance segmentation. We formulate a key instance selection problem on the instance candidate forest as an optimization problem based on the earth mover’s distance (EMD), and solve it by integer linear programming. Extensive experiments on eight biomedical video or 3D datasets demonstrate that H-EMD consistently boosts DL semantic segmentation models and is highly competitive with state-of-the-art methods.

中文翻译：

---

H-EMD：用于实例分割的分层推土机距离方法


基于深度学习（DL）的语义分割方法在生物医学图像分割中取得了优异的性能，产生高质量的概率图以允许提取丰富的实例信息以促进良好的实例分割。尽管人们为开发新的深度学习语义分割模型付出了巨大的努力，但很少有人关注如何有效地探索其概率图以获得最佳实例分割的关键问题。我们观察到，深度学习语义分割模型的概率图可用于生成许多可能的实例候选，并且可以通过从中选择一组“优化”候选作为输出实例来实现准确的实例分割。此外，生成的候选实例形成良好的分层结构（森林），这允许以优化的方式选择实例。因此，我们提出了一种新颖的框架，称为分层推土机距离（H-EMD），例如生物医学 2D+时间视频和 3D 图像中的分割，它明智地将一致的实例选择与语义分割生成的概率图结合起来。 H-EMD包含两个主要阶段：（1）实例候选生成：通过在森林结构中生成许多实例候选来捕获概率图中的实例结构信息； (2)实例候选选择：从候选集中选择实例进行最终的实例分割。我们将实例候选森林上的关键实例选择问题表述为基于推土机距离（EMD）的优化问题，并通过整数线性规划来求解。 对 8 个生物医学视频或 3D 数据集进行的大量实验表明，H-EMD 始终如一地增强了 DL 语义分割模型，并且与最先进的方法具有很强的竞争力。