Face sketch synthesis has a wide range of applications in both digital entertainment and law enforcement. State-of-the-art examplar-based methods typically exploit a Probabilistic Graphical Model (PGM) to represent the joint probability distribution over all of the patches selected from a set of training data. However, these methods suffer from two main shortcomings: (1) most of these methods capture the evidence between patches in pixel-level, which lead to inaccurate parameter estimation under bad environment conditions such as light variations and clutter backgrounds; (2) the assumption that a photo patch and its corresponding sketch patch share similar geometric manifold structure is not rigorous. It has shown that deep convolutional neural network (CNN) has outstanding performance in learning to extract high-level feature representation. Therefore, we extract uniform deep patch representations of test photo patches and training sketch patches from a specially designed CNN model to replace pixel intensity, and directly match between them, which can help select better candidate patches from training data as well as improve parameter learning process. In this way, we investigate a novel face sketch synthesis method called DPGM that combines generative PGM and discriminative deep patch representation, which can jointly model the distribution over the parameters for deep patch representation and the distribution over the parameters for sketch patch reconstruction. Then, we apply an alternating iterative optimization strategy to simultaneously optimize two kinds of parameters. Therefore, both the representation capability of deep patch representation and the reconstruction ability of sketch patches can be boosted. Eventually, high quality reconstructed sketches which is robust against light variations and clutter backgrounds can be obtained. Extensive experiments on several benchmark datasets demonstrate that our method can achieve superior performance than other state-of-the-art methods, especially under the case of bad light conditions or clutter backgrounds.

人脸素描合成在数字娱乐和执法中都有广泛的应用。最先进的基于示例的方法通常利用概率图形模型（PGM）来表示从一组训练数据中选择的所有补丁上的联合概率分布。然而，这些方法存在两个主要缺点：（1）这些方法中的大多数都在像素级补丁之间捕获证据，这导致在恶劣的环境条件下，例如光线变化和背景杂乱，参数估计不准确；（2）假设一个照片补丁及其相应的草图补丁共享相似的几何流形结构的假设并不严格。结果表明，深度卷积神经网络（CNN）在学习提取高级特征表示方面具有出色的性能。因此，我们从专门设计的CNN模型中提取测试照片补丁和训练草图补丁的统一深层补丁表示，以替换像素强度，并在它们之间直接匹配，这有助于从训练数据中选择更好的候选补丁，并改善参数学习过程。通过这种方式，我们研究了一种称为DPGM的新型人脸草图合成方法，该方法将生成的PGM与判别性深层补丁表示相结合，可以共同对用于深层补丁表示的参数分布和用于草图补丁重构的参数分布进行联合建模。然后，我们应用交替迭代优化策略来同时优化两种参数。所以，既可以增强深度补丁的表示能力，又可以提高草图补丁的重构能力。最终，可以获得对光线变化和杂乱背景具有鲁棒性的高质量重建草图。在多个基准数据集上进行的大量实验表明，与其他最新方法相比，我们的方法可以实现更高的性能，尤其是在光线条件较差或背景杂乱的情况下。