Machine learning models, especially neural network (NN) classifiers, have acceptable performance and accuracy that leads to their wide adoption in different aspects of our daily lives. The underlying assumption is that these models are generated and used in attack free scenarios. However, it has been shown that neural network based classifiers are vulnerable to adversarial examples. Adversarial examples are inputs with special perturbations that are ignored by human eyes while can mislead NN classifiers. Most of the existing methods for generating such perturbations require a certain level of knowledge about the target classifier, which makes them not very practical. For example, some generators require knowledge of pre-softmax logits while others utilize prediction scores. In this paper, we design a practical black-box adversarial example generator, dubbed ManiGen. ManiGen does not require any knowledge of the inner state of the target classifier. It generates adversarial examples by searching along the manifold, which is a concise representation of input data. Through extensive set of experiments on different datasets, we show that (1) adversarial examples generated by ManiGen can mislead standalone classifiers by being as successful as the state-of-the-art white-box generator, Carlini, and (2) adversarial examples generated by ManiGen can more effectively attack classifiers with state-of-the-art defenses.

中文翻译：

---

ManiGen：对抗样本的流形辅助黑盒生成器

机器学习模型，尤其是神经网络 (NN) 分类器，具有可接受的性能和准确性，这导致它们在我们日常生活的不同方面得到广泛采用。基本假设是这些模型是在无攻击场景中生成和使用的。然而，已经表明基于神经网络的分类器容易受到对抗性示例的影响。对抗性示例是具有特殊扰动的输入，人眼会忽略这些扰动，但会误导 NN 分类器。大多数现有的产生这种扰动的方法都需要对目标分类器有一定程度的了解，这使得它们不太实用。例如，一些生成器需要 pre-softmax logits 的知识，而其他生成器则使用预测分数。在本文中，我们设计了一个实用的黑盒对抗示例生成器，称为 ManiGen。ManiGen 不需要任何关于目标分类器内部状态的知识。它通过沿着流形搜索生成对抗性示例，流形是输入数据的简明表示。通过在不同数据集上进行的大量实验，我们表明 (1) ManiGen 生成的对抗性示例可以通过与最先进的白盒生成器 Carlini 一样成功来误导独立分类器，以及 (2) 对抗性示例ManiGen 生成的分类器可以通过最先进的防御更有效地攻击分类器。