We address the task of converting a floorplan and a set of associated photos of a residence into a textured 3D mesh model, a task which we call Plan2Scene. Our system 1) lifts a floorplan image to a 3D mesh model; 2) synthesizes surface textures based on the input photos; and 3) infers textures for unobserved surfaces using a graph neural network architecture. To train and evaluate our system we create indoor surface texture datasets, and augment a dataset of floorplans and photos from prior work with rectified surface crops and additional annotations. Our approach handles the challenge of producing tileable textures for dominant surfaces such as floors, walls, and ceilings from a sparse set of unaligned photos that only partially cover the residence. Qualitative and quantitative evaluations show that our system produces realistic 3D interior models, outperforming baseline approaches on a suite of texture quality metrics and as measured by a holistic user study.

中文翻译：

---

Plan2Scene：将平面图转换为 3D 场景

我们解决了将住宅的平面图和一组相关照片转换为带纹理的 3D 网格模型的任务，我们称之为 Plan2Scene 的任务。我们的系统 1) 将平面图图像提升为 3D 网格模型；2）根据输入的照片合成表面纹理；和 3) 使用图神经网络架构推断未观察到的表面的纹理。为了训练和评估我们的系统，我们创建了室内表面纹理数据集，并使用修正的表面裁剪和附加注释来扩充先前工作中的平面图和照片数据集。我们的方法解决了从仅部分覆盖住所的稀疏未对齐照片集为地板、墙壁和天花板等主要表面生成可平铺纹理的挑战。