Recent work has shown great progress in building photorealistic animatable full-body codec avatars, but these avatars still face difficulties in generating high-fidelity animation of clothing. To address the difficulties, we propose a method to build an animatable clothed body avatar with an explicit representation of the clothing on the upper body from multi-view captured videos. We use a two-layer mesh representation to separately register the 3D scans with templates. In order to improve the photometric correspondence across different frames, texture alignment is then performed through inverse rendering of the clothing geometry and texture predicted by a variational autoencoder. We then train a new two-layer codec avatar with separate modeling of the upper clothing and the inner body layer. To learn the interaction between the body dynamics and clothing states, we use a temporal convolution network to predict the clothing latent code based on a sequence of input skeletal poses. We show photorealistic animation output for three different actors, and demonstrate the advantage of our clothed-body avatars over single-layer avatars in the previous work. We also show the benefit of an explicit clothing model which allows the clothing texture to be edited in the animation output.

中文翻译：

---

可动画全身化身的显式服装建模

最近的工作在构建逼真的动画全身编解码器化身方面取得了很大进展，但这些化身在生成高保真服装动画方面仍然面临困难。为了解决这些困难，我们提出了一种方法来构建一个可动画的衣服身体化身，从多视图捕获的视频中明确表示上半身的衣服。我们使用两层网格表示将 3D 扫描与模板分开注册。为了改善不同帧之间的光度对应，然后通过对由变分自动编码器预测的服装几何形状和纹理进行逆向渲染来执行纹理对齐。然后，我们训练一个新的两层编解码器化身，对上衣和身体内层进行单独建模。为了学习身体动力学和服装状态之间的相互作用，我们使用时间卷积网络根据输入骨架姿势序列预测服装潜在代码。我们展示了三个不同演员的逼真动画输出，并在之前的工作中展示了我们的衣身化身相对于单层化身的优势。我们还展示了显式服装模型的好处，它允许在动画输出中编辑服装纹理。