Purpose

Automatic workflow recognition from surgical videos is fundamental and significant for developing context-aware systems in modern operating rooms. Although many approaches have been proposed to tackle challenges in this complex task, there are still many problems such as the fine-grained characteristics and spatial–temporal discrepancies in surgical videos.

Methods

We propose a contrastive learning-based convolutional recurrent network with multi-level prediction to tackle these problems. Specifically, split-attention blocks are employed to extract spatial features. Through a mapping function in the step-phase branch, the current workflow can be predicted on two mutual-boosting levels. Furthermore, a contrastive branch is introduced to learn the spatial–temporal features that eliminate irrelevant changes in the environment.

Results

We evaluate our method on the Cataract-101 dataset. The results show that our method achieves an accuracy of 96.37% with only surgical step labels, which outperforms other state-of-the-art approaches.

Conclusion

The proposed convolutional recurrent network based on step-phase prediction and contrastive learning can leverage fine-grained characteristics and alleviate spatial–temporal discrepancies to improve the performance of surgical workflow recognition.

中文翻译：

---

反对时空差异：基于对比学习的网络，用于手术工作流识别

目的

对于现代手术室中开发情境感知系统而言，从手术视频中自动识别工作流程至关重要。尽管已经提出了许多方法来应对这一复杂任务中的挑战，但是仍然存在许多问题，例如手术录像中的细粒度特征和时空差异。

方法

我们提出了一种基于对比学习的卷积递归网络，该网络具有多级预测，可以解决这些问题。具体而言，采用分割注意块来提取空间特征。通过步骤阶段分支中的映射功能，可以在两个相互提升的层次上预测当前的工作流程。此外，引入了对比分支来学习消除环境无关变化的时空特征。

结果

我们在Cataract-101数据集上评估我们的方法。结果表明，我们的方法仅使用外科手术步骤标签即可达到96.37％的准确性，优于其他最新方法。

结论

提出的基于阶段预测和对比学习的卷积递归网络可以利用细粒度的特征并缓解时空差异，从而提高手术工作流程识别的性能。