当前位置:
X-MOL 学术
›
arXiv.cs.GR
›
论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
GeoSim: Photorealistic Image Simulation with Geometry-Aware Composition
arXiv - CS - Graphics Pub Date : 2021-01-16 , DOI: arxiv-2101.06543 Yun Chen, Frieda Rong, Shivam Duggal, Shenlong Wang, Xinchen Yan, Sivabalan Manivasagam, Shangjie Xue, Ersin Yumer, Raquel Urtasun
arXiv - CS - Graphics Pub Date : 2021-01-16 , DOI: arxiv-2101.06543 Yun Chen, Frieda Rong, Shivam Duggal, Shenlong Wang, Xinchen Yan, Sivabalan Manivasagam, Shangjie Xue, Ersin Yumer, Raquel Urtasun
Scalable sensor simulation is an important yet challenging open problem for
safety-critical domains such as self-driving. Current work in image simulation
either fail to be photorealistic or do not model the 3D environment and the
dynamic objects within, losing high-level control and physical realism. In this
paper, we present GeoSim, a geometry-aware image composition process that
synthesizes novel urban driving scenes by augmenting existing images with
dynamic objects extracted from other scenes and rendered at novel poses.
Towards this goal, we first build a diverse bank of 3D objects with both
realistic geometry and appearance from sensor data. During simulation, we
perform a novel geometry-aware simulation-by-composition procedure which 1)
proposes plausible and realistic object placements into a given scene, 2)
renders novel views of dynamic objects from the asset bank, and 3) composes and
blends the rendered image segments. The resulting synthetic images are
photorealistic, traffic-aware, and geometrically consistent, allowing image
simulation to scale to complex use cases. We demonstrate two such important
applications: long-range realistic video simulation across multiple camera
sensors, and synthetic data generation for data augmentation on downstream
segmentation tasks.
中文翻译:
GeoSim:具有几何感知构图的真实感图像模拟
对于诸如安全驾驶等安全关键领域,可扩展传感器仿真是一个重要而又具有挑战性的开放问题。当前在图像仿真中的工作要么无法实现真实感,要么无法对3D环境及其内部的动态对象进行建模,从而失去了高级控制和物理真实感。在本文中,我们介绍了GeoSim,这是一种几何感知图像合成过程,通过使用从其他场景提取并以新颖姿势渲染的动态对象增强现有图像来合成新颖的城市驾驶场景。为了实现这一目标,我们首先根据传感器数据构建具有逼真的几何形状和外观的3D对象库。在仿真过程中,我们执行了一种新颖的几何构图模拟程序,该程序包括:1)将合理可行的对象放置在给定场景中,2)渲染资产库中动态对象的新颖视图,3)合成并融合渲染的图像段。生成的合成图像具有真实感,可感知流量并且在几何上是一致的,从而可以将图像模拟扩展到复杂的用例。我们演示了两个重要的应用:跨多个摄像头传感器的远程真实视频仿真,以及用于下游分段任务的数据增强的合成数据生成。
更新日期:2021-01-19
中文翻译:
GeoSim:具有几何感知构图的真实感图像模拟
对于诸如安全驾驶等安全关键领域,可扩展传感器仿真是一个重要而又具有挑战性的开放问题。当前在图像仿真中的工作要么无法实现真实感,要么无法对3D环境及其内部的动态对象进行建模,从而失去了高级控制和物理真实感。在本文中,我们介绍了GeoSim,这是一种几何感知图像合成过程,通过使用从其他场景提取并以新颖姿势渲染的动态对象增强现有图像来合成新颖的城市驾驶场景。为了实现这一目标,我们首先根据传感器数据构建具有逼真的几何形状和外观的3D对象库。在仿真过程中,我们执行了一种新颖的几何构图模拟程序,该程序包括:1)将合理可行的对象放置在给定场景中,2)渲染资产库中动态对象的新颖视图,3)合成并融合渲染的图像段。生成的合成图像具有真实感,可感知流量并且在几何上是一致的,从而可以将图像模拟扩展到复杂的用例。我们演示了两个重要的应用:跨多个摄像头传感器的远程真实视频仿真,以及用于下游分段任务的数据增强的合成数据生成。