In this paper, we introduce a novel 3D shape reconstruction method from a single-view sketch image based on a deep neural network. The proposed pipeline is mainly composed of three modules. The first module is sketch component segmentation based on multimodal DNN fusion and is used to segment a given sketch into a series of basic units and build a transformation template by the knots between them. The second module is a nonlinear transformation network for multifarious sketch generation with the obtained transformation template. It creates the transformation representation of a sketch by extracting the shape features of an input sketch and transformation template samples. The third module is deep 3D shape reconstruction using multifarious sketches, which takes the obtained sketches as input to reconstruct 3D shapes with a generative model. It fuses and optimizes features of multiple views and thus is more likely to generate high-quality 3D shapes. To evaluate the effectiveness of the proposed method, we conduct extensive experiments on a public 3D reconstruction dataset. The results demonstrate that our model can achieve better reconstruction performance than peer methods. Specifically, compared to the state-of-the-art method, the proposed model achieves a performance gain in terms of the five evaluation metrics by an average of 25.5% on the man-made model dataset and 23.4% on the character object dataset using synthetic sketches and by an average of 31.8% and 29.5% on the two datasets, respectively, using human drawing sketches.

中文翻译：

---

使用深度神经网络从单视图草图重建3D模型

在本文中，我们介绍了一种基于深度神经网络的单视图素描图像中新颖的3D形状重构方法。拟议中的管道主要由三个模块组成。第一个模块是基于多模态DNN融合的草图成分分割，用于将给定的草图分割为一系列基本单元，并通过它们之间的节点构建变换模板。第二个模块是一个非线性变换网络，用于使用获得的变换模板生成大量草图。它通过提取输入草图的形状特征和变换模板样本来创建草图的变换表示。第三个模块是使用多种草图的深层3D形状重构，该模型将获得的草图作为输入以使用生成模型重建3D形状。它融合并优化了多个视图的功能，因此更有可能生成高质量的3D形状。为了评估该方法的有效性，我们在一个公共的3D重建数据集上进行了广泛的实验。结果表明，与同级方法相比，我们的模型可以实现更好的重建性能。具体而言，与最新方法相比，所提出的模型在五个评估指标方面的性能提升为人工模型数据集平均为25.5％，而使用角色对象数据集则为23.4％人工草图，并且在两个数据集上使用人工草图的平均比例分别为31.8％和29.5％。我们对公共3D重建数据集进行了广泛的实验。结果表明，我们的模型比同等方法可以实现更好的重建性能。具体而言，与最新方法相比，所提出的模型在五个评估指标方面的性能提升为人工模型数据集平均为25.5％，而使用角色对象数据集则为23.4％人工草图，并且在两个数据集上使用人工草图的平均比例分别为31.8％和29.5％。我们对公共3D重建数据集进行了广泛的实验。结果表明，我们的模型比同等方法可以实现更好的重建性能。具体而言，与最新方法相比，所提出的模型在五个评估指标方面的性能提升为人工模型数据集平均为25.5％，而使用角色对象数据集则为23.4％人工草图，并且在两个数据集上使用人工草图的平均比例分别为31.8％和29.5％。