Portrait relighting aims to render a face image under different lighting conditions. Existing methods do not explicitly consider some challenging lighting effects such as specular and shadow, and thus may fail in handling extreme lighting conditions. In this paper, we propose a novel framework that explicitly models multiple reflectance channels for single image portrait relighting, including the facial albedo, geometry as well as two lighting effects, i.e. , specular and shadow. These channels are finally composed to generate the relit results via deep neural networks. Current datasets do not support learning such multiple reflectance channel modeling. Therefore, we present a large-scale dataset with the ground-truths of the channels, enabling us to train the deep neural networks in a supervised manner. Furthermore, we develop a novel module named Lighting guided Feature Modulation (LFM). In contrast to existing methods which simply incorporate the given lighting in the bottleneck of a network, LFM fuses the lighting by layer-wise feature modulation to deliver more convincing results. Extensive experiments demonstrate that our proposed method achieves better results and is able to generate challenging lighting effects.

中文翻译：

---

通过显式多反射通道建模的单图像肖像重新照明

人像重新照明旨在在不同的照明条件下渲染人脸图像。现有方法没有明确考虑一些具有挑战性的光照效果，例如镜面反射和阴影，因此可能无法处理极端光照条件。在本文中，我们提出了一个新颖的框架，该框架显式地为单张图像人像重新照明的多个反射通道建模，包括面部反照率、几何形状以及两种照明效果，IE、镜面反射和阴影。这些通道最终被组合起来，通过深度神经网络生成 relit 结果。当前的数据集不支持学习这种多反射通道建模。因此，我们提出了一个包含通道真实情况的大规模数据集，使我们能够以监督的方式训练深度神经网络。此外，我们开发了一个名为照明引导特征调制 (LFM) 的新型模块。与将给定光照简单地合并到网络瓶颈中的现有方法相比，LFM 通过逐层特征调制来融合光照以提供更令人信服的结果。大量实验表明，我们提出的方法取得了更好的效果，并且能够产生具有挑战性的照明效果。