Depth sensing devices enabled with an RGB camera, can be used to augment conventional images with depth information on a per-pixel basis. Currently available RGB-D sensors include the Asus Xtion Pro, Microsoft Kinect and Intel RealSense™. However, these sensors have certain limitations. Objects that are shiny, transparent or have an absorbing matte surface, create problems due to reflection. Also, there can be an interference in the IR pattern due to the use of multiple RGB-D cameras and the depth information is correctly interpreted only for short distances between the camera and the object. The proposed system, block matching stereo vision (BMSV) uses an RGB-D camera with rectified/non-rectified block matching and image pyramiding along with dynamic programming for human tracking and capture of accurate depth information from shiny/transparent objects. Here, the IR emitter generates a known IR pattern and the depth information is recovered by comparing the multiple views of the focused object. The depth map of the BMSV RGB-D camera and the resultant disparity map are fused. This fills any void regions that may have emerged due to interference or because of the reflective transparent surfaces and an enhanced dense stereo image is obtained. The proposed method is applied to a 3D realistic head model, a functional magnetic resonance image (fMRI) and the results are presented. Results showed an improvement in speed and accuracy of RGB-D sensors which in turn provided accurate depth information density irrespective of the object surface.

中文翻译：

---

使用块匹配和图像金字塔立体视觉启用 RGB-D 传感器的深度信息增强

配备 RGB 相机的深度感测设备可用于以每像素为基础，使用深度信息来增强传统图像。当前可用的 RGB-D 传感器包括 Asus Xtion Pro、Microsoft Kinect 和 Intel RealSense™。然而，这些传感器有一定的局限性。有光泽、透明或具有吸收性哑光表面的物体会因反射而产生问题。此外，由于使用多个 RGB-D 摄像头，IR 模式可能会出现干扰，并且深度信息仅在摄像头和物体之间的短距离内被正确解释。提议的系统，块匹配立体视觉 (BMSV) 使用具有校正/非校正块匹配和图像金字塔化的 RGB-D 相机以及动态编程，用于人类跟踪和从闪亮/透明物体捕获准确的深度信息。在这里，IR 发射器生成已知的 IR 模式，并通过比较聚焦对象的多个视图来恢复深度信息。BMSV RGB-D 相机的深度图和由此产生的视差图被融合。这填充了可能由于干扰或反射透明表面而出现的任何空隙区域，并获得增强的密集立体图像。将所提出的方法应用于 3D 逼真头部模型、功能磁共振图像 (fMRI) 并呈现结果。