Context:

Recently, advances in Deep Learning (DL) have promoted the development of DL-driven image recognition systems in various fields, such as medical treatment, face detection, etc., almost achieving the same level of performance as the human brain. Nevertheless, using DL-driven image recognition systems in these safety-critical domains requires ensuring the accuracy and the stability of these systems. Recent research in this direction mainly focuses on using the image transformations for the overall image to detect the inconsistency of image recognition systems. However, the influence of the image background region ($i.e.$, the region of the image other than the target object) on the recognition result of the systems and the robustness evaluation of the systems are not considered.

Objective:

To evaluate the robustness of DL-driven image recognition systems about image background region changes, this paper introduces DeepBackground, a novel metamorphic testing method for DL-driven image recognition systems.

Method:

First, we define a new metric, termed Background-Relevance (BRC) to assess the influence degree of the image background region on the recognition result of the image recognition systems. DeepBackground defines a series of domain-specific metamorphic relations (MRs) combined with BRC and automatically generates many follow-up test images based on these MRs. Finally, DeepBackground detects the inconsistency of these systems and evaluates their robustness about image background changes according to BRC.

Results:

Our empirical validation on 3 commercial image recognition services and 6 popular convolutional neural networks (CNNs) models shows that DeepBackground can not only evaluate the robustness of these image recognition systems about image background changes according to BRC, but also can detect their inconsistent behaviors.

Conclusion:

DeepBackground is capable of automatically generating high-quality test input images to detect the inconsistency of the image recognition systems, and evaluating the robustness of these systems about image background changes according to BRC.

中文翻译：

---

DeepBackground：伴随背景相关性的深度学习驱动的图像识别系统的变形测试

语境：

最近，深度学习（DL）的进步推动了深度学习驱动的图像识别系统在医疗、人脸检测等各个领域的发展，几乎达到了与人脑相同的性能水平。然而，在这些安全关键领域使用深度学习驱动的图像识别系统需要确保这些系统的准确性和稳定性。最近这个方向的研究主要集中在使用整体图像的图像变换来检测图像识别系统的不一致性。然而，图像背景区域的影响（$\mathrm{一世}.\mathrm{电子}.$, 目标物体以外的图像区域) 对系统的识别结果和系统的鲁棒性评估没有考虑。

客观的：

为了评估DL驱动的图像识别系统对图像背景区域变化的鲁棒性，本文介绍了DeepBackground，一种用于DL驱动的图像识别系统的新型变形测试方法。

方法：

首先，我们定义了一个新的度量，称为背景相关性（BRC）来评估图像背景区域对图像识别系统识别结果的影响程度。DeepBackground 定义了一系列结合 BRC 的特定领域的变形关系（MR），并根据这些 MR 自动生成许多后续测试图像。最后，DeepBackground 检测这些系统的不一致性，并根据 BRC 评估它们对图像背景变化的鲁棒性。

结果：

我们对 3 个商业图像识别服务和 6 个流行的卷积神经网络 (CNN) 模型的实证验证表明，DeepBackground 不仅可以根据 BRC 评估这些图像识别系统对图像背景变化的鲁棒性，还可以检测它们的不一致行为。

结论：

DeepBackground 能够自动生成高质量的测试输入图像，以检测图像识别系统的不一致性，并根据 BRC 评估这些系统对图像背景变化的鲁棒性。