Free navigation of a scene requires warping some reference views to some desired target viewpoint and blending them to synthesize a virtual view. Convolutional Neural Networks (ConvNets) based methods can learn both the warping and blending tasks jointly. Such methods are often designed for moderate inter-camera baseline distance and larger kernels are required for warping if the baseline distance increases. Algorithmic methods can in principle deal with large baselines, however the synthesized view suffers from artifacts near disoccluded pixels. We present a hybrid approach where first, reference views are algorithmically warped to the target position and then are blended via a ConvNet. Preliminary view warping allows reducing the size of the convolutional kernels and thus the learnable parameters count. We propose a residual encoder–decoder for image blending with a Siamese encoder to further keep the parameters count low. We also contribute a hole inpainting algorithm to fill the disocclusions in the warped views. Our view synthesis experiments on real multiview sequences show better objective image quality than state-of-the-art methods due to fewer artifacts in the synthesized images.

场景的自由导航需要将一些参考视图扭曲到一些所需的目标视点并将它们混合以合成虚拟视图。基于卷积神经网络 (ConvNets) 的方法可以联合学习变形和混合任务。这种方法通常是为中等的相机间基线距离而设计的，如果基线距离增加，则需要更大的内核来进行扭曲。算法方法原则上可以处理大基线，但是合成视图会在未遮挡像素附近产生伪影。我们提出了一种混合方法，首先，参考视图在算法上扭曲到目标位置，然后通过 ConvNet 混合。初步视图扭曲允许减少卷积核的大小，从而减少可学习参数的数量。我们提出了一种残差编码器 - 解码器，用于与连体编码器进行图像混合，以进一步保持低参数计数。我们还提供了一个洞修复算法来填充扭曲视图中的遮挡。由于合成图像中的伪影更少，我们对真实多视图序列的视图合成实验显示出比最先进方法更好的客观图像质量。