Window detection is a key component in many graphics and vision applications related to 3D city modeling and scene visualization. We present a novel approach for learning to recognize windows in a colored facade image. Rather than predicting bounding boxes or performing facade segmentation, our system locates keypoints of windows, and learns keypoint relationships to group them together into windows. A further module provides extra recognizable information at the window center. Locations and relationships of keypoints are encoded in different types of heatmaps, which are learned in an end-to-end network. We have also constructed a facade dataset with 3 418 annotated images to facilitate research in this field. It has richly varying facade structures, occlusion, lighting conditions, and angle of view. On our dataset, our method achieves precision of 91.4% and recall of 91.0% under 50% IoU (intersection over union). We also make a quantitative comparison with state-of-the-art methods to verify the utility of our proposed method. Applications based on our window detector are also demonstrated, such as window blending.

中文翻译：

---

使用热图融合进行外墙窗口检测

窗口检测是与 3D 城市建模和场景可视化相关的许多图形和视觉应用中的关键组件。我们提出了一种学习在彩色立面图像中识别窗户的新方法。我们的系统不是预测边界框或执行立面分割，而是定位窗口的关键点，并学习关键点关系以将它们组合成窗口。另一个模块在窗口中心提供额外的可识别信息。关键点的位置和关系被编码在不同类型的热图中，这些热图是在端到端网络中学习的。我们还构建了一个包含 3 418 张带注释的图像的立面数据集，以促进该领域的研究。它具有丰富多样的立面结构、遮挡、照明条件和视角。在我们的数据集上，我们的方法在 50% IoU（交集对联合）下实现了 91.4% 的精度和 91.0% 的召回率。我们还与最先进的方法进行了定量比较，以验证我们提出的方法的实用性。还演示了基于我们的窗口检测器的应用程序，例如窗口混合。