Our aim is to give free-viewpoint photo-realistic rendering of real indoor scenes, using a sparse collection of RGB-D image as input. Image based rendering (IBR) is an effective way to achieve both realism and interactivity. However, there are several challenges for IBR: misalignment of object boundaries between color-and-depth image pairs often leads to ghost contours; projection errors result in the visibility failure; and useless and redundant input views often produce blurring images. To address these issues, we propose a pixel-to-pixel multi-view depth refinement method to produce pixel-accurate alignment between color-and-depth image pairs, and an adaptive view selection approach to avoid choosing redundant or useless input views. Furthermore, we propose a layered 3D warping to handle occluded elements. These components are designed to work together, reducing visual artifacts and providing plausible free-viewpoint synthesized images. The evaluation results indicate that our method achieves good performance on a wide variety of challenging scenes and performs best among popular IBR algorithms designed for dynamic scenes.

我们的目标是使用 RGB-D 图像的稀疏集合作为输入，提供真实室内场景的自由视角照片般逼真渲染。基于图像的渲染 (IBR) 是实现真实感和交互性的有效方式。然而，IBR 有几个挑战：颜色和深度图像对之间的对象边界未对齐通常会导致重影轮廓；投影错误导致能见度失败；无用和冗余的输入视图通常会产生模糊的图像。为了解决这些问题，我们提出了一种像素到像素的多视图深度细化方法，以在颜色和深度图像对之间产生像素精确的对齐，以及一种自适应视图选择方法，以避免选择冗余或无用的输入视图。此外，我们提出了一种分层的 3D 扭曲来处理被遮挡的元素。这些组件旨在协同工作，减少视觉伪影并提供可信的自由视点合成图像。评估结果表明，我们的方法在各种具有挑战性的场景上都取得了良好的性能，并且在为动态场景设计的流行 IBR 算法中表现最好。