UI design is an integral part of software development. For many developers who do not have much UI design experience, exposing them to a large database of real-application UI designs can help them quickly build up a realistic understanding of the design space for a software feature and get design inspirations from existing applications. However, existing keyword-based, image-similarity-based, and component-matching-based methods cannot reliably find relevant high-fidelity UI designs in a large database alike to the UI wireframe that the developers sketch, in face of the great variations in UI designs. In this article, we propose a deep-learning-based UI design search engine to fill in the gap. The key innovation of our search engine is to train a wireframe image autoencoder using a large database of real-application UI designs, without the need for labeling relevant UI designs. We implement our approach for Android UI design search, and conduct extensive experiments with artificially created relevant UI designs and human evaluation of UI design search results. Our experiments confirm the superior performance of our search engine over existing image-similarity or component-matching-based methods and demonstrate the usefulness of our search engine in real-world UI design tasks.

中文翻译：

---

通过图像自动编码器进行基于线框的 UI 设计搜索

UI设计是软件开发的一个组成部分。对于许多没有太多 UI 设计经验的开发人员来说，让他们接触到真实应用程序 UI 设计的大型数据库可以帮助他们快速建立对软件功能设计空间的真实理解，并从现有应用程序中获得设计灵感。然而，现有的基于关键字、基于图像相似性和基于组件匹配的方法无法可靠地在大型数据库中找到相关的高保真 UI 设计，例如开发人员绘制的 UI 线框，面对巨大的变化。用户界面设计。在本文中，我们提出了一个基于深度学习的 UI 设计搜索引擎来填补这一空白。我们搜索引擎的关键创新是使用真实应用程序 UI 设计的大型数据库来训练线框图像自动编码器，无需标记相关的 UI 设计。我们实现了我们的 Android UI 设计搜索方法，并通过人工创建的相关 UI 设计和对 UI 设计搜索结果的人工评估进行了广泛的实验。我们的实验证实了我们的搜索引擎优于现有的基于图像相似性或组件匹配的方法，并证明了我们的搜索引擎在现实世界的 UI 设计任务中的有用性。