There has recently been great interest in neural rendering methods. Some approaches use 3D geometry reconstructed with Multi-View Stereo (MVS) but cannot recover from the errors of this process, while others directly learn a volumetric neural representation, but suffer from expensive training and inference. We introduce a general approach that is initialized with MVS, but allows further optimization of scene properties in the space of input views, including depth and reprojected features, resulting in improved novel-view synthesis. A key element of our approach is our new differentiable point-based pipeline, based on bi-directional Elliptical Weighted Average splatting, a probabilistic depth test and effective camera selection. We use these elements together in our neural renderer, that outperforms all previous methods both in quality and speed in almost all scenes we tested. Our pipeline can be applied to multi-view harmonization and stylization in addition to novel-view synthesis.

中文翻译：

---

具有逐视图优化的基于点的神经渲染

最近人们对神经渲染方法产生了极大的兴趣。一些方法使用多视图立体 (MVS) 重建的 3D 几何，但无法从该过程的错误中恢复，而另一些方法则直接学习体积神经表示，但需要进行昂贵的训练和推理。我们介绍了一种用 MVS 初始化的通用方法，但允许进一步优化输入视图空间中的场景属性，包括深度和重投影特征，从而改进新视图合成。我们方法的一个关键元素是我们新的基于可微分的管道，基于双向椭圆加权平均 splatting、概率深度测试和有效的相机选择。我们在神经渲染器中一起使用这些元素，在我们测试的几乎所有场景中，它在质量和速度方面都优于所有以前的方法。除了新视图合成之外，我们的管道还可以应用于多视图协调和风格化。