We propose a novel shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection (ESD) surgery. This task is of great clinical significance but extremely challenging due to bleeding, lighting reflection, and motion blur in the complicated surgical environment. Compared with existing solutions, which either neglect geometric relationships among targeting objects or capture the relationships by using complicated aggregation schemes, the proposed network is capable of achieving satisfactory accuracy while maintaining real-time performance by taking full advantage of the spatial relations among landmarks. We first devise an algorithm to automatically generate relation keypoint heatmaps, which are able to intuitively represent the prior knowledge of spatial relations among landmarks without using any extra manual annotation efforts. We then develop two complementary regularization schemes to progressively incorporate the prior knowledge into the training process. While one scheme introduces pixel-level regularization by multi-task learning, the other integrates global-level regularization by harnessing a newly designed grouped consistency evaluator, which adds relation constraints to the proposed network in an adversarial manner. Both schemes are beneficial to the model in training, and can be readily unloaded in inference to achieve real-time detection. We establish a large in-house dataset of ESD surgery for esophageal cancer to validate the effectiveness of our proposed method. Extensive experimental results demonstrate that our approach outperforms state-of-the-art methods in terms of accuracy and efficiency, achieving better detection results faster. Promising results on two downstream applications further corroborate the great potential of our method in ESD clinical practice.

我们提出了一种新颖的形状感知关系网络，用于在内窥镜黏膜下剥离术 (ESD) 手术中进行准确和实时的地标检测。这项任务具有重要的临床意义，但由于复杂的手术环境中的出血、光线反射和运动模糊，极具挑战性。与忽略目标对象之间的几何关系或使用复杂的聚合方案捕获关系的现有解决方案相比，所提出的网络能够充分利用地标之间的空间关系，在保持实时性能的同时达到令人满意的精度。我们首先设计了一种算法来自动生成关系关键点热图，它能够直观地表示地标之间空间关系的先验知识，而无需使用任何额外的手动注释工作。然后，我们开发了两个互补的正则化方案，以逐步将先验知识纳入训练过程。一种方案通过多任务学习引入像素级正则化，另一种方案通过利用新设计的分组一致性评估器集成全局级正则化，以对抗方式为提议的网络添加关系约束。两种方案在训练中都有利于模型，在推理中可以很容易地卸载，实现实时检测。我们建立了食管癌 ESD 手术的大型内部数据集，以验证我们提出的方法的有效性。大量的实验结果表明，我们的方法在准确性和效率方面优于最先进的方法，更快地获得更好的检测结果。两个下游应用的有希望的结果进一步证实了我们的方法在 ESD 临床实践中的巨大潜力。