In this paper, we focus on the problem of novel view synthesis for vehicles. Some previous works solve the problem of novel view synthesis in a controlled 3D environment by exploiting additional 3D details ( i.e. , camera viewpoints and underlying 3D models). However, in real scenarios, the 3D details are difficult to obtain. In this case, we find that introducing vehicle pose to represent the views of vehicles is an alternative paradigm to solve the lack of 3D details. In novel view synthesis, preserving local details is one of the most challenging problems. To address this problem, we propose a perspective-aware generative model (PAGM). We are motivated by the prior that vehicles are made of quadrilateral planes. Preserving these rigid planes during image generation ensures that image details are kept. To this end, a classic image transformation method is leveraged, i.e. , perspective transformation. In our GAN-based system, the perspective transformation is applied to the encoder feature maps, and the resulting maps are regarded as new conditions for the decoder. This strategy preserves the quadrilateral planes all the way through the network, thus shuttling the texture details from the input image to the generated image. In the experiments, we show that PAGM can generate high-quality vehicle images with fine details. Quantitatively, our method is superior to several competing approaches employing either GAN or the perspective transformation. Code is available at: https://github.com/ilvkai/view-synthesis-for-vehicles

中文翻译：

---

车辆基于姿势的视图综合：一种透视感知方法

在本文中，我们关注于车辆的新颖视图综合问题。先前的一些工作通过利用其他3D细节解决了在受控3D环境中新颖的视图合成问题（ 即 ，相机视点和基础3D模型）。但是，在实际情况下，很难获得3D细节。在这种情况下，我们发现引入车辆姿态来表示车辆的视图是解决缺少3D细节的替代范式。在新颖的视图合成中，保留局部细节是最具挑战性的问题之一。为了解决这个问题，我们提出了一种透视感知的生成模型（PAGM）。以前，车辆是由四边形平面制成的，这给我们带来了动力。在图像生成期间保留这些刚性平面可确保保留图像细节。为此，采用了经典的图像转换方法，即 ，透视变换。在我们基于GAN的系统中，将透视变换应用于编码器特征图，并将生成的图视为解码器的新条件。该策略将四边形平面一直保留到整个网络，从而将纹理细节从输入图像穿梭到生成的图像。在实验中，我们表明PAGM可以生成具有精细细节的高质量车辆图像。从数量上讲，我们的方法优于采用GAN或透视变换的几种竞争方法。代码位于：https://github.com/ilvkai/view-synthesis-for-vehicles