We study the notion of consistency between a 3D shape and a 2D observation and propose a differentiable formulation which allows computing gradients of the 3D shape given an observation from an arbitrary view. We do so by reformulating view consistency using a differentiable ray consistency (DRC) term. We show that this formulation can be incorporated in a learning framework to leverage different types of multi-view observations e.g., foreground masks, depth, color images, semantics etc. as supervision for learning single-view 3D prediction. We present empirical analysis of our technique in a controlled setting. We also show that this approach allows us to improve over existing techniques for single-view reconstruction of objects from the PASCAL VOC dataset.

中文翻译：

---

基于可微光线一致性的单视图重建的多视图监督

我们研究了 3D 形状和 2D 观察之间的一致性概念，并提出了一种可微分公式，该公式允许在给定任意视图观察的情况下计算 3D 形状的梯度。我们通过使用可微光线一致性 (DRC) 术语重新制定视图一致性来做到这一点。我们表明，该公式可以合并到学习框架中，以利用不同类型的多视图观察，例如前景蒙版、深度、彩色图像、语义等，作为学习单视图 3D 预测的监督。我们在受控环境中对我们的技术进行了实证分析。我们还表明，这种方法使我们能够改进现有技术，用于从 PASCAL VOC 数据集中重建对象的单视图。