The initial analysis of any large data set can be divided into two phases: (1) the identification of common trends or patterns and (2) the identification of anomalies or outliers that deviate from those trends. We focus on the goal of detecting observations with novel content, which can alert us to artifacts in the data set or, potentially, the discovery of previously unknown phenomena. To aid in interpreting and diagnosing the novel aspect of these selected observations, we recommend the use of novelty detection methods that generate explanations. In the context of large image data sets, these explanations should highlight what aspect of a given image is new (color, shape, texture, content) in a human-comprehensible form. We propose DEMUD-VIS, the first method for providing visual explanations of novel image content by employing a convolutional neural network (CNN) to extract image features, a method that uses reconstruction error to detect novel content, and an up-convolutional network to convert CNN feature representations back into image space. We demonstrate this approach on diverse images from ImageNet, freshwater streams, and the surface of Mars. Finally, we evaluate the utility of the visual explanations with a user study.

任何大型数据集的初始分析可以分为两个阶段：（1）识别常见趋势或模式，以及（2）识别偏离那些趋势的异常或异常值。我们专注于检测具有新颖内容的观测结果的目标，这可以使我们警惕数据集中的伪像，或者有可能发现以前未知的现象。为了帮助解释和诊断这些选定观察的新颖性，我们建议使用产生解释的新颖性检测方法。在大型图像数据集的上下文中，这些解释应突出显示以人类可理解的形式显示的给定图像的哪个方面是新的（颜色，形状，纹理，内容）。我们建议DEMUD-VIS，第一种通过使用卷积神经网络（CNN）提取图像特征来提供新颖图像内容的视觉解释的方法，一种使用重建误差检测新颖内容的方法以及一种向上卷积网络将CNN特征表示转换回图像的方法空间。我们在ImageNet，淡水流和火星表面的各种图像上演示了这种方法。最后，我们通过用户研究评估了视觉解释的效用。