Many operations in robot-assisted surgery (RAS) can be viewed in a hierarchical manner. Each surgical task is represented by a superstate, which can be decomposed into finer-grained states. The estimation of these discrete states at different levels of temporal granularity provides a temporal perception of the current surgical scene during RAS, which is a crucial step towards many automated surgeon-assisting functionalities. We propose Hierarchical Estimation of Surgical States through Deep Neural Networks (HESS-DNN), a deep learning-based system that concurrently estimates the current super- and fine-grained states. HESS-DNN incorporates endoscopic vision, robot kinematics, and system events data from the da Vinci Xi surgical system. HESS-DNN is evaluated on a real-world robotic inguinal hernia repair surgery dataset: HERNIA-20, and achieves accurate state estimates of both surgical superstate and the corresponding fine-grained surgical state. We show that HESS-DNN improves state-of-the-art fine-grained state estimation across the entire HERNIA-20 RAS procedure through its hierarchical design. We also analyze the relative contributions of each input data type and HESS-DNN's design to surgical (super)state estimation accuracy.

中文翻译：

---

通过深度神经网络在机器人辅助手术期间进行自主分层手术状态估计

机器人辅助手术 (RAS) 中的许多操作可以以分层方式进行查看。每个手术任务都由一个超状态表示，它可以分解为更细粒度的状态。在不同时间粒度级别对这些离散状态的估计提供了 RAS 期间当前手术场景的时间感知，这是实现许多自动化外科医生辅助功能的关键一步。我们建议通过深度神经网络 (HESS-DNN) 对手术状态进行分层估计，这是一种基于深度学习的系统，可同时估计当前的超细粒度状态。HESS-DNN 结合了来自 da Vinci Xi 手术系统的内窥镜视觉、机器人运动学和系统事件数据。HESS-DNN 在真实世界的机器人腹股沟疝修补手术数据集上进行评估：HERNIA-20，并实现手术超状态和相应的细粒度手术状态的准确状态估计。我们表明，HESS-DNN 通过其分层设计改进了整个 HERNIA-20 RAS 程序中最先进的细粒度状态估计。我们还分析了每种输入数据类型和 HESS-DNN 设计对手术（超）状态估计精度的相对贡献。