当前位置:
X-MOL 学术
›
arXiv.cs.SE
›
论文详情
Our official English website, www.x-mol.net, welcomes your
feedback! (Note: you will need to create a separate account there.)
Robustness Evaluation of Stacked Generative Adversarial Networks using Metamorphic Testing
arXiv - CS - Software Engineering Pub Date : 2021-03-04 , DOI: arxiv-2103.02870 Hyejin Park, Taaha Waseem, Wen Qi Teo, Ying Hwei Low, Mei Kuan Lim, Chun Yong Chong
arXiv - CS - Software Engineering Pub Date : 2021-03-04 , DOI: arxiv-2103.02870 Hyejin Park, Taaha Waseem, Wen Qi Teo, Ying Hwei Low, Mei Kuan Lim, Chun Yong Chong
Synthesising photo-realistic images from natural language is one of the
challenging problems in computer vision. Over the past decade, a number of
approaches have been proposed, of which the improved Stacked Generative
Adversarial Network (StackGAN-v2) has proven capable of generating high
resolution images that reflect the details specified in the input text
descriptions. In this paper, we aim to assess the robustness and
fault-tolerance capability of the StackGAN-v2 model by introducing variations
in the training data. However, due to the working principle of Generative
Adversarial Network (GAN), it is difficult to predict the output of the model
when the training data are modified. Hence, in this work, we adopt Metamorphic
Testing technique to evaluate the robustness of the model with a variety of
unexpected training dataset. As such, we first implement StackGAN-v2 algorithm
and test the pre-trained model provided by the original authors to establish a
ground truth for our experiments. We then identify a metamorphic relation, from
which test cases are generated. Further, metamorphic relations were derived
successively based on the observations of prior test results. Finally, we
synthesise the results from our experiment of all the metamorphic relations and
found that StackGAN-v2 algorithm is susceptible to input images with obtrusive
objects, even if it overlaps with the main object minimally, which was not
reported by the authors and users of StackGAN-v2 model. The proposed
metamorphic relations can be applied to other text-to-image synthesis models to
not only verify the robustness but also to help researchers understand and
interpret the results made by the machine learning models.
中文翻译:
基于变质检验的堆叠式生成对抗网络的鲁棒性评估
从自然语言合成逼真的图像是计算机视觉中的难题之一。在过去的十年中,已经提出了许多方法,其中改进的堆叠生成对抗网络(StackGAN-v2)已被证明能够生成高分辨率图像,这些图像反映输入文本描述中指定的细节。在本文中,我们旨在通过在训练数据中引入变化来评估StackGAN-v2模型的鲁棒性和容错能力。但是,由于生成对抗网络(GAN)的工作原理,修改训练数据时很难预测模型的输出。因此,在这项工作中,我们采用变质测试技术来评估具有各种意外训练数据集的模型的鲁棒性。因此,我们首先实现StackGAN-v2算法并测试原始作者提供的预训练模型,以为我们的实验建立基础。然后,我们确定一个变质关系,从中生成测试用例。此外,基于先前测试结果的观察结果,相继得出了变形关系。最后,我们综合了所有变质关系实验的结果,发现StackGAN-v2算法即使在与主要对象的重叠最小的情况下,也容易受到带有突出物体的输入图像的影响,这并未被作者和用户报道。 StackGAN-v2模型。
更新日期:2021-03-05
中文翻译:
基于变质检验的堆叠式生成对抗网络的鲁棒性评估
从自然语言合成逼真的图像是计算机视觉中的难题之一。在过去的十年中,已经提出了许多方法,其中改进的堆叠生成对抗网络(StackGAN-v2)已被证明能够生成高分辨率图像,这些图像反映输入文本描述中指定的细节。在本文中,我们旨在通过在训练数据中引入变化来评估StackGAN-v2模型的鲁棒性和容错能力。但是,由于生成对抗网络(GAN)的工作原理,修改训练数据时很难预测模型的输出。因此,在这项工作中,我们采用变质测试技术来评估具有各种意外训练数据集的模型的鲁棒性。因此,我们首先实现StackGAN-v2算法并测试原始作者提供的预训练模型,以为我们的实验建立基础。然后,我们确定一个变质关系,从中生成测试用例。此外,基于先前测试结果的观察结果,相继得出了变形关系。最后,我们综合了所有变质关系实验的结果,发现StackGAN-v2算法即使在与主要对象的重叠最小的情况下,也容易受到带有突出物体的输入图像的影响,这并未被作者和用户报道。 StackGAN-v2模型。