Human imitation has become topical recently, driven by GAN's ability to disentangle human pose and body content. However, the latest methods hardly focus on 3D information, and to avoid self-occlusion, a massive amount of input images are needed. In this paper, we propose RIN, a novel volume-based framework for reconstructing a textured 3D model from a single picture and imitating a subject with the generated model. Specifically, to estimate most of the human texture, we propose a U-Net-like front-to-back translation network. With both front and back images input, the textured volume recovery module allows us to color a volumetric human. A sequence of 3D poses then guides the colored volume via Flowable Disentangle Networks as a volume-to-volume translation task. To project volumes to a 2D plane during training, we design a differentiable depth-aware renderer. Our experiments demonstrate that our volume-based model is adequate for human imitation, and the back view can be estimated reliably using our network. While prior works based on either 2D pose or semantic map often fail for the unstable appearance of a human, our framework can still produce concrete results, which are competitive to those imagined from multi-view input.

中文翻译：

---

RIN：具有单个图像的纹理化人体模型恢复和模仿

在GAN分解人类姿势和身体内容的能力的驱使下，模仿人类已成为热门话题。但是，最新的方法几乎不关注3D信息，并且为了避免自我遮挡，需要大量的输入图像。在本文中，我们提出了RIN，这是一种新颖的基于体积的框架，用于从单个图片重建纹理3D模型并使用生成的模型模仿对象。具体来说，为了估计大多数人的纹理，我们提出了类似U-Net的前后翻译网络。通过输入正反两面的图像，带纹理的体积恢复模块使我们能够为体积人体着色。然后，一系列3D姿势通过Flowable Disentangle网络引导有色体积作为体积到体积的翻译任务。要在训练期间将体积投影到2D平面上，我们设计了一个可区分的深度感知渲染器。我们的实验表明，基于体积的模型足以用于人体模仿，并且可以使用我们的网络可靠地估算后视图。尽管基于2D姿势或语义图的先前工作通常会因人的不稳定外观而失败，但我们的框架仍可以产生具体的结果，与多视图输入所想象的结果相比具有竞争力。