Manual visual inspection is the most common means of assessing the condition of civil infrastructure in the United States, but can be exceedingly laborious, time-consuming, and dangerous. Research has focused on automating parts of the inspection process using unmanned aerial vehicles for image acquisition, followed by deep learning techniques for damage identification. Existing deep learning methods and datasets for inspections have typically been developed for a single damage type. However, most guidelines for inspections require the identification of multiple damage types and describe evaluating the significance of the damage based on the associated material type. Thus, the identification of material type is important in understanding the meaning of the identified damage. Training separate networks for the tasks of material and damage identification fails to incorporate this intrinsic interdependence between them. We hypothesize that a network that incorporates such interdependence directly will have a better accuracy in material and damage identification. To this end, a deep neural network, termed the material-and-damage-network (MaDnet), is proposed to simultaneously identify material type (concrete, steel, asphalt), as well as fine (cracks, exposed rebar) and coarse (spalling, corrosion) structural damage. In this approach, semantic segmentation (i.e., assignment of each pixel in the image with a material and damage label) is employed, where the interdependence between material and damage is incorporated through shared filters learned through multi-objective optimization. A new dataset with pixel-level labels identifying the material and damage type is developed and made available to the research community. Finally, the dataset is used to evaluate MaDnet and demonstrate the improvement in pixel accuracy over employing independent networks.

手动外观检查是评估美国民用基础设施状况的最常用方法，但可能非常费力，费时且危险。研究重点是使用无人机进行图像采集，然后是深度学习技术以进行损伤识别，从而使部分检查过程自动化。现有的用于检查的深度学习方法和数据集通常是针对单个损坏类型开发的。但是，大多数检查准则要求识别多种损坏类型，并根据相关的材料类型描述评估损坏的重要性。因此，材料类型的识别对于理解所识别的损坏的含义很重要。训练单独的网络来执行物料和损坏识别的任务无法将它们之间的这种内在相互依存结合在一起。我们假设直接包含这种相互依赖性的网络在材料和损坏识别方面将具有更好的准确性。为此，提出了一种称为材料和损伤网络（MaDnet）的深层神经网络，以同时识别材料类型（混凝土，钢，沥青）以及精细（裂缝，裸露钢筋）和粗糙材料（剥落，腐蚀）结构损坏。在这种方法中，采用语义分割（即，在图像中为每个像素分配材质和损坏标签），其中，材质和损坏之间的相互依赖性通过通过多目标优化学习的共享过滤器合并。开发了一个新的数据集，该数据集具有标识材料和损坏类型的像素级标签，并可供研究团体使用。最终，该数据集用于评估MaDnet并证明了像素精度优于采用独立网络的情况。