In this paper, we present an attribute grammar for solving two coupled tasks: i) parsing a 2D image into semantic regions; and ii) recovering the 3D scene structures of all regions. The proposed grammar consists of a set of production rules, each describing a kind of spatial relation between planar surfaces in 3D scenes. These production rules are used to decompose an input image into a hierarchical parse graph representation where each graph node indicates a planar surface or a composite surface. Different from other stochastic image grammars, the proposed grammar augments each graph node with a set of attribute variables to depict scene-level globalgeometry, e.g., camera focal length, or local geometry, e.g., surface normal, contact lines between surfaces. These geometric attributes impose constraints between a node and its off-springs in the parse graph. Under a probabilistic framework, we develop a Markov Chain Monte Carlo method to construct a parse graph that optimizes the 2D image recognition and 3D scene reconstruction purposes simultaneously. We evaluated our method on both public benchmarks and newly collected datasets. Experiments demonstrate that the proposed method is capable of achieving state-of-the-art scene reconstruction of a single image.

中文翻译：

---

单视图 3D 场景重构和属性语法解析


在本文中，我们提出了一种用于解决两个耦合任务的属性语法：i）将 2D 图像解析为语义区域； ii) 恢复所有区域的 3D 场景结构。所提出的语法由一组产生规则组成，每个规则描述 3D 场景中平面之间的一种空间关系。这些产生式规则用于将输入图像分解为分层解析图表示，其中每个图节点指示平面或复合表面。与其他随机图像语法不同，所提出的语法用一组属性变量增强每个图节点，以描述场景级全局几何形状，例如相机焦距，或局部几何形状，例如表面法线、表面之间的接触线。这些几何属性在解析图中的节点及其后代之间施加了约束。在概率框架下，我们开发了马尔可夫链蒙特卡罗方法来构建解析图，同时优化 2D 图像识别和 3D 场景重建目的。我们根据公共基准和新收集的数据集评估了我们的方法。实验表明，所提出的方法能够实现单幅图像最先进的场景重建。