3D shape reconstruction from multiple hand-drawn sketches is an intriguing way to 3D shape modeling. Currently, state-of-the-art methods employ neural networks to learn a mapping from multiple sketches from arbitrary view angles to a 3D voxel grid. Because of the cubic complexity of 3D voxel grids, however, neural networks are hard to train and limited to low resolution reconstructions, which leads to a lack of geometric detail and low accuracy. To resolve this issue, we propose to reconstruct 3D shapes from multiple sketches using direct shape optimization (DSO), which does not involve deep learning models for direct voxel-based 3D shape generation. Specifically, we first leverage a conditional generative adversarial network (CGAN) to translate each sketch into an attenuance image that captures the predicted geometry from a given viewpoint. Then, DSO minimizes a project-and-compare loss to reconstruct the 3D shape such that it matches the predicted attenuance images from the view angles of all input sketches. Based on this, we further propose a progressive update approach to handle inconsistencies among a few hand-drawn sketches for the same 3D shape. Our experimental results show that our method significantly outperforms the state-of-the-art methods under widely used benchmarks and produces intuitive results in an interactive application.

中文翻译：

---

使用直接形状优化从多个草图重构3D形状。

从多个手绘草图进行3D形状重建是3D形状建模的一种有趣方式。当前，最先进的方法采用神经网络来学习从任意角度的多个草图到3D体素网格的映射。但是，由于3D体素网格的立方复杂性，神经网络很难训练，并且仅限于低分辨率重建，这导致缺少几何细节和准确性低。为了解决此问题，我们建议使用直接形状优化（DSO）从多个草图中重建3D形状，该方法不涉及用于直接基于体素的3D形状生成的深度学习模型。具体来说，我们首先利用条件生成对抗网络（CGAN）将每个草图转换为衰减图像，该衰减图像从给定视点捕获预测的几何形状。然后，DSO将投影和比较损失最小化以重建3D形状，从而使其与所有输入草图的视角匹配的预测衰减图像。基于此，我们进一步提出了一种渐进式更新方法，以处理同一3D形状的一些手绘草图之间的不一致。我们的实验结果表明，在广泛使用的基准下，我们的方法明显优于最新方法，并在交互式应用程序中产生直观的结果。