We perform a comprehensive study on the performance of derivative free optimization (DFO) algorithms for the generation of targeted black-box adversarial attacks on Deep Neural Network (DNN) classifiers assuming the perturbation energy is bounded by an \(\ell _\infty\) constraint and the number of queries to the network is limited. This paper considers four pre-existing state-of-the-art DFO-based algorithms along with a further developed algorithm built on BOBYQA, a model-based DFO method. We compare these algorithms in a variety of settings according to the fraction of images that they successfully misclassify given a maximum number of queries to the DNN. The experiments disclose how the likelihood of finding an adversarial example depends on both the algorithm used and the setting of the attack; algorithms limiting the search of adversarial example to the vertices of the \(\ell ^\infty\) constraint work particularly well without structural defenses, while the presented BOBYQA based algorithm works better for especially small perturbation energies. This variance in performance highlights the importance of new algorithms being compared to the state-of-the-art in a variety of settings, and the effectiveness of adversarial defenses being tested using as wide a range of algorithms as possible.

我们对无导数优化 (DFO) 算法的性能进行了全面研究，以生成对深度神经网络 (DNN) 分类器的有针对性的黑盒对抗性攻击，假设扰动能量以\(\ell _\infty\ )约束并且对网络的查询数量是有限的。本文考虑了四种预先存在的最先进的基于 DFO 的算法以及建立在 BOBYQA（一种基于模型的 DFO 方法）上的进一步开发的算法。我们在给定最大数量的 DNN 查询的情况下，根据它们成功错误分类的图像的比例，在各种设置中比较这些算法。实验揭示了找到对抗样本的可能性如何取决于所使用的算法和攻击的设置；算法将对抗样本的搜索限制在\(\ell ^\infty\)的顶点约束在没有结构防御的情况下工作得特别好，而基于 BOBYQA 的算法在特别小的扰动能量下工作得更好。这种性能差异凸显了新算法在各种设置中与最先进算法进行比较的重要性，以及使用尽可能广泛的算法测试对抗性防御的有效性。