Sketch‐based modeling aims to recover three‐dimensional (3D) shape from two‐dimensional line drawings. However, due to the sparsity and ambiguity of the sketch, it is extremely challenging for computers to interpret line drawings of physical objects. Most conventional systems are restricted to specific scenarios such as recovering for specific shapes, which are not conducive to generalize. Recent progress of deep learning methods have sparked new ideas for solving computer vision and pattern recognition issues. In this work, we present an end‐to‐end learning framework to predict 3D shape from line drawings. Our approach is based on a two‐steps strategy, it converts the sketch image to its normal image, then recover the 3D shape subsequently. A differentiable renderer is proposed and incorporated into this framework, it allows the integration of the rendering pipeline with neural networks. Experimental results show our method outperforms the state‐of‐art, which demonstrates that our framework is able to cope with the challenges in single sketch‐based 3D shape modeling.

中文翻译：

---

具有可微渲染器的基于草图的建模

基于草图的建模旨在从二维线条图中恢复三维 (3D) 形状。然而，由于草图的稀疏性和歧义性，计算机解释物理对象的线条图极具挑战性。大多数常规系统仅限于特定场景，例如恢复特定形状，不利于泛化。深度学习方法的最新进展激发了解决计算机视觉和模式识别问题的新思路。在这项工作中，我们提出了一个端到端的学习框架来从线条图中预测 3D 形状。我们的方法基于两步策略，它将草图图像转换为其正常图像，然后随后恢复 3D 形状。提出了一种可微渲染器并将其合并到该框架中，它允许将渲染管道与神经网络集成。实验结果表明我们的方法优于最先进的方法，这表明我们的框架能够应对基于单个草图的 3D 形状建模中的挑战。