Land-use classification based on spaceborne or aerial remote sensing images has been extensively studied over the past decades. Such classification is usually a patch-wise or pixel-wise labeling over the whole image. But for many applications, such as urban population density mapping or urban utility planning, a classification map based on individual buildings is much more informative. However, such semantic classification still poses some fundamental challenges, for example, how to retrieve fine boundaries of individual buildings. In this paper, we proposed a general framework for classifying the functionality of individual buildings. The proposed method is based on Convolutional Neural Networks (CNNs) which classify façade structures from street view images, such as Google StreetView, in addition to remote sensing images which usually only show roof structures. Geographic information was utilized to mask out individual buildings, and to associate the corresponding street view images. We created a benchmark dataset which was used for training and evaluating CNNs. In addition, the method was applied to generate building classification maps on both region and city scales of several cities in Canada and the US.

在过去的几十年中，已经广泛研究了基于星载或航空遥感影像的土地利用分类。这种分类通常是对整个图像进行逐块或逐像素标记。但是对于许多应用程序（例如城市人口密度图或城市公用事业规划），基于单个建筑物的分类图更具参考价值。但是，这种语义分类仍然带来一些基本挑战，例如，如何检索单个建筑物的精细边界。在本文中，我们提出了用于对单个建筑物的功能进行分类的通用框架。所提出的方法基于卷积神经网络（CNN），可根据街景图像（例如Google StreetView）对立面结构进行分类，除了通常仅显示屋顶结构的遥感图像。地理信息被用来掩盖单个建筑物，并关联相应的街景图像。我们创建了一个基准数据集，用于训练和评估CNN。此外，该方法还用于生成加拿大和美国几个城市的区域和城市规模的建筑物分类图。