Depth maps obtained through sensors are often unsatisfactory because of their low-resolution and noise interference. In this paper, we propose a real-time depth map enhancement system based on a residual network which uses dual channels to process depth maps and intensity maps respectively and cancels the preprocessing process, and the algorithm proposed can achieve real-time processing speed at more than 30 fps. Furthermore, the FPGA design and implementation for depth sensing is also introduced. In this FPGA design, intensity image and depth image are captured by the dual-camera synchronous acquisition system as the input of neural network. Experiments on various depth map restoration shows our algorithms has better performance than existing LRMC, DE-CNN and DDTF algorithms on standard datasets and has a better depth map super-resolution, and our FPGA completed the test of the system to ensure that the data throughput of the USB 3.0 interface of the acquisition system is stable at 226 Mbps, and support dual-camera to work at full speed, that is, 54 (1280 × 960 + 328 × 248 × 3). View Full-Text

中文翻译：

---

基于残差网络和FPGA的实时深度图增强系统

通过传感器获得的深度图通常由于分辨率低和噪声干扰而不能令人满意。本文提出了一种基于残差网络的实时深度图增强系统，该系统利用双通道分别处理深度图和强度图，并取消了预处理过程，所提出的算法可以更快地实现实时处理速度。超过30 fps。此外，还介绍了用于深度感测的FPGA设计和实现。在此FPGA设计中，双摄像机同步采集系统捕获强度图像和深度图像作为神经网络的输入。在各种深度图还原上进行的实验表明，我们的算法在标准数据集上的性能优于现有的LRMC，DE-CNN和DDTF算法，并且具有更好的深度图超分辨率，我们的FPGA完成了系统测试，以确保采集系统的USB 3.0接口的数据吞吐量稳定在226 Mbps，并支持双摄像头全速工作，即54（1280×960 + 328×248×3）。查看全文