In this paper, we propose a multi-frame depth super-resolution (SR) method based on $L_{1}$ data fidelity with the total variation regularization (TV- $L_{1}$ ) model. The majority of time-of-flight (ToF) sensors exhibit limited spatial resolution compared to RGB sensors and the improvement of the depth image resolution is an inherently ill-posed problem. To overcome this under-determined problem, the solution space is limited by the regularization term through prior knowledge and the data fidelity term using statistical information of the noise. Firstly, the statistical characteristics of ToF depth images are analyzed to specify the appropriate observation model. Thereafter, the objective function for multi-frame depth SR based on the TV- $L_{1}$ model is designed by considering the prior knowledge of the depth images. This approach enables the sharpness of the edges to be preserved and the noise amplification to be suppressed simultaneously. Furthermore, an efficient solver based on half-quadratic splitting is proposed. The algorithm minimizes the objective function for the multi-frame SR problem consisting of the TV regularization term and $L_{1}$ data fidelity term. The proposed method is verified on a synthetic dataset and real-world data acquired from a ToF sensor. The experimental results demonstrate that the proposed method can substantially reconstruct high-resolution depth images in terms of preserving sharp depth discontinuities, without any obvious artifacts, and can increase robustness to noise.

中文翻译：

---

具有总变异正则化 L1 函数的 ToF 传感器的多帧深度超分辨率

在本文中，我们提出了一种基于多帧深度超分辨率（SR）的方法 $L_{1}$ 总变异正则化的数据保真度（TV- $L_{1}$ ） 模型。与 RGB 传感器相比，大多数飞行时间 (ToF) 传感器的空间分辨率有限，深度图像分辨率的提高是一个固有的不适定问题。为了克服这个不确定的问题，解决方案空间受到通过先验知识的正则化项和使用噪声统计信息的数据保真度项的限制。首先，分析ToF深度图像的统计特征，指定合适的观测模型。此后，多帧深度 SR 的目标函数基于 TV- $L_{1}$ 模型是通过考虑深度图像的先验知识来设计的。这种方法能够同时保持边缘的锐度并抑制噪声放大。此外，提出了一种基于半二次分裂的高效求解器。该算法最小化多帧 SR 问题的目标函数，该问题由 TV 正则化项和 $L_{1}$ 数据保真度术语。所提出的方法在合成数据集和从 ToF 传感器获取的真实世界数据上得到验证。实验结果表明，所提出的方法可以在保留尖锐的深度不连续性方面基本上重建高分辨率深度图像，没有任何明显的伪影，并且可以增加对噪声的鲁棒性。