In the past few years, Generative Adversarial Networks (GANs) have dramatically advanced our ability to represent and parameterize high-dimensional, non-linear image manifolds. As a result, they have been widely adopted across a variety of applications, ranging from challenging inverse problems like image completion, to problems such as anomaly detection and adversarial defense. A recurring theme in many of these applications is the notion of projecting an image observation onto the manifold that is inferred by the generator. In this context, Projected Gradient Descent (PGD) has been the most popular approach, which essentially optimizes for a latent vector that minimizes the discrepancy between a generated image and the given observation. However, PGD is a brittle optimization technique that fails to identify the right projection (or latent vector) when the observation is corrupted, or perturbed even by a small amount. Such corruptions are common in the real world, for example images in the wild come with unknown crops, rotations, missing pixels, or other kinds of non-linear distributional shifts which break current encoding methods, rendering downstream applications unusable. To address this, we propose corruption mimicking—a new robust projection technique, that utilizes a surrogate network to approximate the unknown corruption directly at test time, without the need for additional supervision or data augmentation. The proposed method is significantly more robust than PGD and other competing methods under a wide variety of corruptions, thereby enabling a more effective use of GANs in real-world applications. More importantly, we show that our approach produces state-of-the-art performance in several GAN-based applications—anomaly detection, domain adaptation, and adversarial defense, that benefit from an accurate projection.

中文翻译：

---

MimicGAN：使用损坏模拟对图像流形进行鲁棒投影

在过去几年中，生成对抗网络 (GAN) 极大地提高了我们表示和参数化高维非线性图像流形的能力。因此，它们已被广泛应用于各种应用，从具有挑战性的逆问题（如图像完成）到异常检测和对抗性防御等问题。在许多这些应用中反复出现的主题是将图像观察投影到由生成器推断的流形上的概念。在这种情况下，投影梯度下降 (PGD) 一直是最流行的方法，它本质上优化了一个潜在向量，最大限度地减少了生成图像和给定观察之间的差异。然而，PGD​​ 是一种脆弱的优化技术，当观察被破坏或受到少量干扰时，它无法识别正确的投影（或潜在向量）。此类损坏在现实世界中很常见，例如，野外图像带有未知的裁剪、旋转、像素丢失或其他类型的非线性分布偏移，这会破坏当前的编码方法，导致下游应用程序无法使用。为了解决这个问题，我们提出了腐败模仿——一种新的鲁棒投影技术，它利用代理网络在测试时直接近似未知的腐败，而无需额外的监督或数据增强。在各种损坏情况下，所提出的方法明显比 PGD 和其他竞争方法更稳健，从而能够在实际应用中更有效地使用 GAN。更重要的是，我们表明我们的方法在几个基于 GAN 的应用程序中产生了最先进的性能——异常检测、域适应和对抗性防御，这些都得益于准确的投影。