Background and objective: Recovering high-quality 3D point clouds from monocular endoscopic images is a challenging task. This paper proposes a novel deep learning-based computational framework for 3D point cloud reconstruction from single monocular endoscopic images.

Methods: An unsupervised mono-depth learning network is used to generate depth information from monocular images. Given a single mono endoscopic image, the network is capable of depicting a depth map. The depth map is then used to recover a dense 3D point cloud. A generative Endo-AE network based on an auto-encoder is trained to repair defects of the dense point cloud by generating the best representation from the incomplete data. The performance of the proposed framework is evaluated against state-of-the-art learning-based methods. The results are also compared with non-learning based stereo 3D reconstruction algorithms.

Results: Our proposed methods outperform both the state-of-the-art learning-based and non-learning based methods for 3D point cloud reconstruction. The Endo-AE model for point cloud completion can generate high-quality, dense 3D endoscopic point clouds from incomplete point clouds with holes. Our framework is able to recover complete 3D point clouds with the missing rate of information up to 60%. Five large medical in-vivo databases of 3D point clouds of real endoscopic scenes have been generated and two synthetic 3D medical datasets are created. We have made these datasets publicly available for researchers free of charge.

Conclusions: The proposed computational framework can produce high-quality and dense 3D point clouds from single mono-endoscopy images for augmented reality, virtual reality and other computer-mediated medical applications.

背景和目的：从单眼内窥镜图像中恢复高质量3D点云是一项艰巨的任务。本文提出了一种基于深度学习的新颖计算框架，用于从单眼内窥镜图像重建3D点云。

方法：使用无监督的单深度学习网络从单眼图像生成深度信息。给定单个单幅内窥镜图像，该网络能够描绘深度图。然后使用深度图恢复密集的3D点云。通过自动编码器生成的Endo-AE网络经过训练，可以通过从不完整数据中生成最佳表示来修复密集点云的缺陷。所提出的框架的性能是根据最新的基于学习的方法进行评估的。还将结果与基于非学习的立体声3D重建算法进行比较。

结果：我们提出的方法优于3D点云重构的最新学习方法和非学习方法。用于点云完成的Endo-AE模型可以从带有孔的不完整点云中生成高质量，密集的3D内窥镜点云。我们的框架能够以高达60％的信息丢失率恢复完整的3D点云。已经生成了五个真实内窥镜场景的3D点云的大型医学体内数据库，并创建了两个合成的3D医学数据集。我们已经免费向研究人员公开提供了这些数据集。

结论：所提出的计算框架可以从单幅内窥镜图像中生成高质量和密集的3D点云，以用于增强现实，虚拟现实和其他计算机介导的医疗应用。