PURPOSE Fetoscopic laser photocoagulation is a minimally invasive surgery for the treatment of twin-to-twin transfusion syndrome (TTTS). By using a lens/fibre-optic scope, inserted into the amniotic cavity, the abnormal placental vascular anastomoses are identified and ablated to regulate blood flow to both fetuses. Limited field-of-view, occlusions due to fetus presence and low visibility make it difficult to identify all vascular anastomoses. Automatic computer-assisted techniques may provide better understanding of the anatomical structure during surgery for risk-free laser photocoagulation and may facilitate in improving mosaics from fetoscopic videos. METHODS We propose FetNet, a combined convolutional neural network (CNN) and long short-term memory (LSTM) recurrent neural network architecture for the spatio-temporal identification of fetoscopic events. We adapt an existing CNN architecture for spatial feature extraction and integrated it with the LSTM network for end-to-end spatio-temporal inference. We introduce differential learning rates during the model training to effectively utilising the pre-trained CNN weights. This may support computer-assisted interventions (CAI) during fetoscopic laser photocoagulation. RESULTS We perform quantitative evaluation of our method using 7 in vivo fetoscopic videos captured from different human TTTS cases. The total duration of these videos was 5551 s (138,780 frames). To test the robustness of the proposed approach, we perform 7-fold cross-validation where each video is treated as a hold-out or test set and training is performed using the remaining videos. CONCLUSION FetNet achieved superior performance compared to the existing CNN-based methods and provided improved inference because of the spatio-temporal information modelling. Online testing of FetNet, using a Tesla V100-DGXS-32GB GPU, achieved a frame rate of 114 fps. These results show that our method could potentially provide a real-time solution for CAI and automating occlusion and photocoagulation identification during fetoscopic procedures.

中文翻译：

---

FetNet：一种用于胎儿视镜视频中遮挡识别的循环卷积网络。


目的 胎儿镜激光光凝术是一种用于治疗双胎输血综合征 (TTTS) 的微创手术。通过使用插入羊膜腔的透镜/光纤镜，可以识别并消融异常的胎盘血管吻合，以调节流向两个胎儿的血流。有限的视野、胎儿存在导致的遮挡和低能见度使得识别所有血管吻合变得困难。自动计算机辅助技术可以在无风险激光光凝手术期间更好地了解解剖结构，并有助于改善胎儿镜视频的马赛克。方法 我们提出 FetNet，一种组合卷积神经网络 (CNN) 和长短期记忆 (LSTM) 循环神经网络架构，用于胎儿视镜事件的时空识别。我们采用现有的 CNN 架构进行空间特征提取，并将其与 LSTM 网络集成以进行端到端时空推理。我们在模型训练期间引入差异学习率，以有效利用预训练的 CNN 权重。这可能支持胎儿镜激光光凝期间的计算机辅助干预（CAI）。结果我们使用从不同人类 TTTS 病例中捕获的 7 个体内胎儿镜视频对我们的方法进行定量评估。这些视频的总时长为 5551 秒（138,780 帧）。为了测试所提出方法的稳健性，我们执行 7 倍交叉验证，其中每个视频被视为保留或测试集，并使用剩余视频进行训练。 结论 与现有的基于 CNN 的方法相比，FetNet 实现了卓越的性能，并且由于时空信息建模而提供了改进的推理。使用 Tesla V100-DGXS-32GB GPU 进行 FetNet 在线测试，帧速率达到 114 fps。这些结果表明，我们的方法有可能为胎儿镜检查过程中的 CAI 和自动闭塞和光凝识别提供实时解决方案。