Owing to the rise of living standard, people attach greater importance to personal appearance, especially clothes matching. With image processing and machine learning technology, we can analyze the pattern of clothes matching for recommendation on clothes images. However, we still face great challenges. To be more specific, there exist excessive complicated factors influencing relation among clothes items, such as color or material, and we also struggle against the problem about how to extract efficient and accurate features. Thus, with the purpose of dealing with such challenges, this paper proposes an efficient clothes matching scheme with Siamese Network and AutoEncoder based on both labeled data from dataset FashionVC and unlabeled data from MicroBlog. More specifically, at first, except for clothes suiting with text from FashionVC, the gallery data also include matching clothes outfits recommended by fashionista in MicroBlog (MbFashion). Meanwhile, a semi-supervised clustering based on assembling was also proposed to generate negative samples to form a comprehensive dataset. Secondly, with consideration of matching patterns from MbFashion, we promoted the Siamese Network properly to more efficiently extract vision features on the constructed training dataset. After that, the traditional features are also extracted, while the Triple AutoEncoder and Bayesian Personalized Ranking are used to map the three kinds of features into the same latent space to learn the compatibility between tops and bottoms. Finally, we conducted a series of experiments and evaluated our results to demonstrate the usefulness and effectiveness of the whole scheme on FashionVC and MbFashion.

中文翻译：

---

基于Siamese Network和AutoEncoder的时尚服饰搭配方案

随着生活水平的提高，人们越来越重视个人的外表，尤其是衣服的搭配。借助图像处理和机器学习技术，我们可以分析衣服匹配的模式，从而对衣服图像进行推荐。然而，我们仍然面临着巨大的挑战。更具体地说，影响衣服项目之间关系的因素存在过多复杂的因素，例如颜色或材料，我们也在努力解决如何高效准确地提取特征的问题。因此，为了应对这些挑战，本文基于来自数据集 FashionVC 的标记数据和来自微博的未标记数据，提出了一种使用 Siamese Network 和 AutoEncoder 的高效服装匹配方案。更具体地说，一开始，除了搭配FashionVC文字的衣服，画廊数据还包括时尚达人在微博（MbFashion）中推荐的搭配服装。同时，还提出了基于组装的半监督聚类来生成负样本以形成综合数据集。其次，考虑到 MbFashion 的匹配模式，我们适当地提升了 Siamese Network，以更有效地在构建的训练数据集上提取视觉特征。之后，也提取了传统特征，同时使用Triple AutoEncoder和Bayesian Personalized Ranking将三种特征映射到相同的潜在空间中，以学习top和bottom之间的兼容性。最后，我们进行了一系列实验并评估了我们的结果，以证明整个方案在 FashionVC 和 MbFashion 上的有用性和有效性。