Manually authoring 3D shapes is difficult and time consuming; generative models of 3D shapes offer compelling alternatives. Procedural representations are one such possibility: they offer high-quality and editable results but are difficult to author and often produce outputs with limited diversity. On the other extreme are deep generative models: given enough data, they can learn to generate any class of shape but their outputs have artifacts and the representation is not editable. In this paper, we take a step towards achieving the best of both worlds for novel 3D shape synthesis. We propose ShapeAssembly, a domain-specific "assembly-language" for 3D shape structures. ShapeAssembly programs construct shapes by declaring cuboid part proxies and attaching them to one another, in a hierarchical and symmetrical fashion. Its functions are parameterized with free variables, so that one program structure is able to capture a family of related shapes. We show how to extract ShapeAssembly programs from existing shape structures in the PartNet dataset. Then we train a deep generative model, a hierarchical sequence VAE, that learns to write novel ShapeAssembly programs. The program captures the subset of variability that is interpretable and editable. The deep model captures correlations across shape collections that are hard to express procedurally. We evaluate our approach by comparing shapes output by our generated programs to those from other recent shape structure synthesis models. We find that our generated shapes are more plausible and physically-valid than those of other methods. Additionally, we assess the latent spaces of these models, and find that ours is better structured and produces smoother interpolations. As an application, we use our generative model and differentiable program interpreter to infer and fit shape programs to unstructured geometry, such as point clouds.

中文翻译：

---

ShapeAssembly：学习为 3D 形状结构合成生成程序

手动创作 3D 形状既困难又耗时；3D 形状的生成模型提供了引人注目的替代方案。程序表示就是这样一种可能性：它们提供高质量和可编辑的结果，但难以创作，并且经常产生多样性有限的输出。另一个极端是深度生成模型：给定足够的数据，它们可以学习生成任何类型的形状，但它们的输出有伪影并且表示不可编辑。在本文中，我们迈出了实现新型 3D 形状合成两全其美的一步。我们提出了 ShapeAssembly，这是一种用于 3D 形状结构的特定领域的“汇编语言”。ShapeAssembly 程序通过声明长方体部件代理并将它们以分层和对称的方式相互连接来构造形状。它的函数是用自由变量参数化的，因此一个程序结构能够捕获一系列相关的形状。我们展示了如何从 PartNet 数据集中现有的形状结构中提取 ShapeAssembly 程序。然后我们训练一个深度生成模型，一个分层序列 VAE，它学习编写新颖的 ShapeAssembly 程序。该程序捕获可解释和可编辑的可变性子集。深度模型捕获了难以在程序上表达的形状集合之间的相关性。我们通过将生成的程序输出的形状与其他最近的形状结构合成模型的形状输出进行比较来评估我们的方法。我们发现我们生成的形状比其他方法更合理且物理上有效。此外，我们评估这些模型的潜在空间，并发现我们的结构更好并产生更平滑的插值。作为一个应用程序，我们使用我们的生成模型和可微分程序解释器来推断形状程序并将其拟合到非结构化几何体，例如点云。