Numerous applications on human faces hinge on depth information. Often, facial stereo matching provides an opportunity to estimate disparity without active projectors. However, existing algorithms are less effective at night due to unclear texture and severe noises in RGB images. In this paper, we address this problem by estimating facial disparity maps from NIR-RGB pairs. We develop a neural network composed of a multi-spectral transfer network (MSTN) and a disparity estimation network (DEN). MSTN is used to produce a pseudo-NIR image aligned with the RGB view using a spatially weighted sum on the NIR one by a kernel prediction network (KPN). As the pseudo-NIR and the NIR images share the same appearance, the facial disparity map is predicted by the proposed DEN with the same-spectral stereo pair. The whole network can be trained in an end-to-end manner and the experimental results demonstrate that it performs favorably against state-of-the-art algorithms on both synthetic and real data.

人脸的众多应用取决于深度信息。通常，面部立体匹配可在没有有源投影仪的情况下提供估算视差的机会。但是，由于RGB图像中的纹理不清晰和严重噪点，现有算法在夜间效果较差。在本文中，我们通过从NIR-RGB对估计面部视差图来解决此问题。我们开发了由多光谱传输网络（MSTN）和视差估计网络（DEN）组成的神经网络。MSTN用于通过内核预测网络（KPN）在NIR上使用空间加权总和生成与RGB视图对齐的伪NIR图像。由于伪近红外图像和近红外图像具有相同的外观，因此建议的DEN使用相同光谱的立体声对预测面部视差图。