In this work, we explore the problems of recommending and visualising makeup products based on images of customers. Focusing on mascara, we propose a two-stage approach that first recommends products to a new customer based on the preferences of other customers with similar visual appearance and then visualises how the recommended products might look on the customer. For the initial product recommendation, we train a Siamese convolutional neural network, using our own dataset of cropped eye regions from images of 91 female subjects, such that it learns to output feature vectors that place images of the same subject close together in high-dimensional space. We evaluate the trained network based on its ability to correctly identify existing subjects from unseen images, and then assess its capability to identify visually similar subjects when an image of a new subject is used as input. For product visualisation, we train per-product generative adversarial networks to map the appearance of a specific product onto an image of a customer with no makeup. We train models to generate images of two mascara formulations and assess their capability to generate realistic mascara lashes while changing as little as possible within non-lash image regions and simulating the different effects of the two products used.

中文翻译：

---

卷积网络，用于基于外观的睫毛膏产品推荐和可视化

在这项工作中，我们探讨了根据客户图像推荐和可视化彩妆产品的问题。针对睫毛膏，我们提出了一种分为两个阶段的方法，该方法首先根据具有相似视觉外观的其他客户的偏好向新客户推荐产品，然后形象化推荐产品在客户身上的外观。对于最初的产品推荐，我们使用我们自己的91个女性受试者的图像中的裁剪眼睛区域数据集训练了一个暹罗卷积神经网络，从而使它学会了输出将相同受试者的图像在高维度上靠在一起的特征向量空间。我们会根据训练有素的网络从看不见的图像中正确识别现有主体的能力来评估该网络，然后在将新主题的图像用作输入时评估其识别视觉相似主题的能力。为了使产品可视化，我们训练了每个产品的生成对抗网络，以将特定产品的外观映射到没有化妆的客户图像上。我们训练模型以生成两种睫毛膏配方的图像，并评估它们生成逼真的睫毛膏睫毛的能力，同时在非睫毛图像区域内尽可能少地改变并模拟所用两种产品的不同效果。