This paper proposes a novel panoramic texture mapping-based rendering system for real-time, photorealistic reproduction of large-scale urban scenes at a street level. Various image-based rendering (IBR) methods have recently been employed to synthesize high-quality novel views, although they require an excessive number of adjacent input images or detailed geometry just to render local views. While the development of global data, such as Google Street View, has accelerated interactive IBR techniques for urban scenes, such methods have hardly been aimed at high-quality street-level rendering. To provide users with free walk-through experiences in global urban streets, our system effectively covers large-scale scenes by using sparsely sampled panoramic street-view images and simplified scene models, which are easily obtainable from open databases. Our key concept is to extract semantic information from the given street-view images and to deploy it in proper intermediate steps of the suggested pipeline, which results in enhanced rendering accuracy and performance time. Furthermore, our method supports real-time semantic 3D inpainting to handle occluded and untextured areas, which appear often when the user's viewpoint dynamically changes. Experimental results validate the effectiveness of this method in comparison with the state-of-the-art approaches. We also present real-time demos in various urban streets.

中文翻译：

---

具有语义对象匹配的即时全景纹理贴图，可用于大规模的城市场景再现

本文提出了一种新颖的基于全景纹理贴图的渲染系统，用于实时，真实地再现街道一级的大型城市场景。尽管已使用大量基于图像的渲染（IBR）方法来渲染局部视图，但是它们需要过多数量的相邻输入图像或详细的几何图形，但这些方法已用于合成高质量的新颖视图。虽然诸如Google Street View之类的全球数据的开发已加速了针对城市场景的交互式IBR技术，但这些方法几乎没有针对高质量的街道级渲染。为了向用户提供在全球城市街道上的免费漫游体验，我们的系统通过使用稀疏采样的全景街景图像和简化的场景模型（可从开放式数据库轻松获得）有效地覆盖了大型场景。我们的关键概念是从给定的街景图像中提取语义信息，并将其部署到建议的管道的适当中间步骤中，从而提高了渲染的准确性和性能。此外，我们的方法支持实时语义3D修复，以处理遮挡和未纹理化的区域，这些区域在用户的视点动态变化时经常出现。与最新技术相比，实验结果证明了该方法的有效性。我们还将在各种城市街道上演示实时演示。我们的方法支持实时语义3D修补，以处理遮挡和无纹理的区域，这些区域通常在用户的视点动态变化时出现。与最新技术相比，实验结果证明了该方法的有效性。我们还将在各种城市街道上演示实时演示。我们的方法支持实时语义3D修补，以处理遮挡和无纹理的区域，这些区域通常在用户的视点动态变化时出现。与最新技术相比，实验结果证明了该方法的有效性。我们还将在各种城市街道上演示实时演示。