In post-event reconnaissance missions, engineers and researchers collect perishable information about damaged buildings in the affected geographical region to learn from the consequences of the event. A typical post-event reconnaissance mission is conducted by first doing a preliminary survey, followed by a detailed survey. The preliminary survey is typically conducted by driving slowly along a pre-determined route, observing the damage, and noting where further detailed data should be collected. This involves several manual, time-consuming steps that can be accelerated by exploiting recent advances in computer vision and artificial intelligence. The objective of this work is to develop and validate an automated technique to support post-event reconnaissance teams in the rapid collection of reliable and sufficiently comprehensive data, for planning the detailed survey. The technique incorporates several methods designed to automate the process of categorizing buildings based on their key physical attributes, and rapidly assessing their post-event structural condition. It is divided into pre-event and post-event streams, each intending to first extract all possible information about the target buildings using both pre-event and post-event images. Algorithms based on convolutional neural network (CNNs) are implemented for scene (image) classification. A probabilistic approach is developed to fuse the results obtained from analyzing several images to yield a robust decision regarding the attributes and condition of a target building. We validate the technique using post-event images captured during reconnaissance missions that took place after hurricanes Harvey and Irma. The validation data were collected by a structural wind and coastal engineering reconnaissance team, the National Science Foundation (NSF) funded Structural Extreme Events Reconnaissance (StEER) Network.

中文翻译：

---

走向全自动的事后数据收集和分析：事前和事后信息融合

在事后侦察任务中，工程师和研究人员收集有关受影响地理区域受损建筑物的易腐烂信息，以了解事件的后果。典型的事后侦察任务是先进行初步调查，然后进行详细调查。初步调查通常是通过沿着预先确定的路线缓慢行驶、观察损坏情况并注意应该在哪里收集更详细的数据来进行的。这涉及几个手动、耗时的步骤，可以通过利用计算机视觉和人工智能的最新进展来加速。这项工作的目标是开发和验证一种自动化技术，以支持事后侦察小组快速收集可靠且足够全面的数据，用于规划详细调查。该技术结合了多种方法，旨在根据建筑物的关键物理属性自动对建筑物进行分类，并快速评估其事后结构状况。它分为事件前和事件后流，每个流都打算首先使用事件前和事件后图像提取有关目标建筑物的所有可能信息。基于卷积神经网络 (CNN) 的算法用于场景（图像）分类。开发了一种概率方法来融合从分析多个图像中获得的结果，以产生关于目标建筑物的属性和条件的稳健决策。我们使用在飓风 Harvey 和 Irma 之后发生的侦察任务期间捕获的事后图像验证了该技术。