We introduce a differential visual similarity metric to train deep neural networks for 3D reconstruction, aimed at improving reconstruction quality. The metric compares two 3D shapes by measuring distances between multi-view images differentiably rendered from the shapes. Importantly, the image-space distance is also differentiable and measures visual similarity, rather than pixel-wise distortion. Specifically, the similarity is defined by mean-squared errors over HardNet features computed from probabilistic keypoint maps of the compared images. Our differential visual shape similarity metric can be easily plugged into various 3D reconstruction networks, replacing their distortion-based losses, such as Chamfer or Earth Mover distances, so as to optimize the network weights to produce reconstructions with better structural fidelity and visual quality. We demonstrate this both objectively, using well-known shape metrics for retrieval and classification tasks that are independent from our new metric, and subjectively through a perceptual study.

中文翻译：

---

DR-KFS：用于 3D 形状重建的可微视觉相似性度量

我们引入了一种差分视觉相似性度量来训练深度神经网络进行 3D 重建，旨在提高重建质量。该指标通过测量从形状不同地渲染的多视图图像之间的距离来比较两个 3D 形状。重要的是，图像空间距离也是可微分的，可以测量视觉相似性，而不是像素级失真。具体来说，相似性是由从比较图像的概率关键点图计算的 HardNet 特征的均方误差定义的。我们的差分视觉形状相似度度量可以轻松插入各种 3D 重建网络，取代它们基于失真的损失，例如倒角或地球移动距离，从而优化网络权重以产生具有更好结构保真度和视觉质量的重建。我们客观地证明了这一点，使用众所周知的形状度量来进行独立于我们的新度量的检索和分类任务，并通过感知研究主观地证明这一点。