We present a neural-based model for relighting a half-body portrait image by simply referring to another portrait image with the desired lighting condition. Rather than following classical inverse rendering methodology that involves estimating normals, albedo and environment maps, we implicitly encode the subject and lighting in a latent space, and use these latent codes to generate relighted images by neural rendering. A key technical innovation is the use of a novel overcomplete lighting representation, which facilitates lighting interpolation in the latent space, as well as helping regularize the self-organization of the lighting latent space during training. In addition, we propose a novel multiplicative neural render that more effectively combines the subject and lighting latent codes for rendering. We also created a large-scale photorealistic rendered relighting dataset for training, which allows our model to generalize well to real images. Extensive experiments demonstrate that our system not only outperforms existing methods for referral-based portrait relighting, but also has the capability generate sequences of relighted images via lighting rotations.

中文翻译：

---

半身人像重新照明与过度完整的照明表示

我们提出了一个基于神经的模型，通过简单地参考另一个具有所需照明条件的人像图像来重新点亮半身人像图像。我们没有遵循涉及估计法线、反照率和环境贴图的经典逆渲染方法，而是在潜在空间中隐式编码主题和照明，并使用这些潜在代码通过神经渲染生成重新照明的图像。一个关键的技术创新是使用了一种新颖的过完备照明表示，它促进了潜在空间中的照明插值，并有助于在训练期间规范照明潜在空间的自组织。此外，我们提出了一种新颖的乘法神经渲染，可以更有效地结合主体和照明潜在代码进行渲染。我们还创建了一个用于训练的大规模逼真渲染重新照明数据集，这使我们的模型能够很好地推广到真实图像。大量实验表明，我们的系统不仅优于现有的基于推荐的人像重新照明方法，而且还具有通过照明旋转生成重新照明图像序列的能力。