One major goal of vision is to infer physical models of objects, surfaces, and their layout from sensors. In this paper, we aim to interpret indoor scenes from one RGBD image. Our representation encodes the layout of orthogonal walls and the extent of objects, modeled with CAD-like 3D shapes. We parse both the visible and occluded portions of the scene and all observable objects, producing a complete 3D parse. Such a scene interpretation is useful for robotics and visual reasoning, but difficult to produce due to the well-known challenge of segmentation, the high degree of occlusion, and the diversity of objects in indoor scenes. We take a data-driven approach, generating sets of potential object regions, matching to regions in training images, and transferring and aligning associated 3D models while encouraging fit to observations and spatial consistency. We use support inference to aid interpretation and propose a retrieval scheme that uses convolutional neural networks to classify regions and retrieve objects with similar shapes. We demonstrate the performance of our method on our newly annotated NYUd v2 dataset (Silberman et al., in: Computer vision-ECCV, 2012, pp 746–760, 2012) with detailed 3D shapes.

中文翻译：

---

从 RGBD 图像完成 3D 场景解析

视觉的一个主要目标是从传感器推断物体、表面及其布局的物理模型。在本文中，我们的目标是从一张 RGBD 图像中解释室内场景。我们的表示对正交墙的布局和对象的范围进行编码，并使用类似 CAD 的 3D 形状建模。我们解析场景的可见部分和被遮挡部分以及所有可观察对象，生成完整的 3D 解析。这种场景解释对机器人和视觉推理很有用，但由于众所周知的分割挑战、高度遮挡以及室内场景中物体的多样性而难以产生。我们采用数据驱动的方法，生成潜在对象区域集，与训练图像中的区域匹配，并转移和对齐相关的 3D 模型，同时鼓励适应观察和空间一致性。我们使用支持推理来帮助解释，并提出了一种检索方案，该方案使用卷积神经网络对区域进行分类并检索具有相似形状的对象。我们在新注释的 NYUd v2 数据集（Silberman 等人，在：Computer vision-ECCV, 2012, pp 746–760, 2012）上展示了我们的方法的性能，其中包含详细的 3D 形状。