We present Magic Layouts; a method for parsing screenshots or hand-drawn sketches of user interface (UI) layouts. Our core contribution is to extend existing detectors to exploit a learned structural prior for UI designs, enabling robust detection of UI components; buttons, text boxes and similar. Specifically we learn a prior over mobile UI layouts, encoding common spatial co-occurrence relationships between different UI components. Conditioning region proposals using this prior leads to performance gains on UI layout parsing for both hand-drawn UIs and app screenshots, which we demonstrate within the context an interactive application for rapidly acquiring digital prototypes of user experience (UX) designs.

中文翻译：

---

魔术布局：用户界面设计中组件检测的结构先验

我们展示魔术布局；一种用于解析用户界面 (UI) 布局的屏幕截图或手绘草图的方法。我们的核心贡献是扩展现有的检测器，以利用已学习的结构先验进行 UI 设计，从而实现对 UI 组件的稳健检测；按钮、文本框等。具体来说，我们学习了移动 UI 布局的先验，编码了不同 UI 组件之间的常见空间共现关系。使用此先验条件调整区域建议可以提高手绘 UI 和应用程序屏幕截图的 UI 布局解析性能，我们在上下文中演示了一个交互式应用程序，用于快速获取用户体验 (UX) 设计的数字原型。