We study the query-based attack against image retrieval to evaluate its robustness against adversarial examples under the black-box setting, where the adversary only has query access to the top-k ranked unlabeled images from the database. Compared with query attacks in image classification, which produce adversaries according to the returned labels or confidence score, the challenge becomes even more prominent due to the difficulty in quantifying the attack effectiveness on the partial retrieved list. In this paper, we make the first attempt in Query-based Attack against Image Retrieval (QAIR), to completely subvert the top-k retrieval results. Specifically, a new relevance-based loss is designed to quantify the attack effects by measuring the set similarity on the top-k retrieval results before and after attacks and guide the gradient optimization. To further boost the attack efficiency, a recursive model stealing method is proposed to acquire transferable priors on the target model and generate the prior-guided gradients. Comprehensive experiments show that the proposed attack achieves a high attack success rate with few queries against the image retrieval systems under the black-box setting. The attack evaluations on the real-world visual search engine show that it successfully deceives a commercial system such as Bing Visual Search with 98% attack success rate by only 33 queries on average.

中文翻译：

---

QAIR：实用的查询有效的黑匣子攻击进行图像检索

我们研究了基于查询的针对图像检索的攻击，以评估其在黑盒设置下针对对抗示例的鲁棒性，其中对手仅具有对数据库中排名前k位未标记图像的查询访问权限。与图像分类中根据返回的标签或置信度分数产生对手的查询攻击相比，由于难以量化部分检索列表上的攻击效果，因此挑战变得更加突出。在本文中，我们首次尝试了基于查询的图像检索攻击（QAIR），以完全颠覆top-k检索结果。具体来说，一种新的基于相关性的损失旨在通过测量攻击前后top-k检索结果的相似度来量化攻击效果，并指导梯度优化。为了进一步提高攻击效率，提出了一种递归模型窃取方法，以获取目标模型上的可转移先验并生成先验指导梯度。综合实验表明，在黑盒设置下，针对图像检索系统的查询很少，所提出的攻击实现了较高的攻击成功率。实际视觉搜索引擎上的攻击评估表明，它成功欺骗了商业系统，例如Bing Visual Search，平均仅33个查询，其攻击成功率达到了98％。提出了一种递归模型窃取方法，以获取目标模型上的可转移先验并生成先验指导梯度。综合实验表明，在黑盒设置下，针对图像检索系统的查询很少，所提出的攻击实现了较高的攻击成功率。实际视觉搜索引擎上的攻击评估表明，它成功欺骗了商业系统，例如Bing Visual Search，平均仅33个查询，其攻击成功率达到了98％。提出了一种递归模型窃取方法，以获取目标模型上的可转移先验并生成先验指导梯度。综合实验表明，在黑盒设置下，针对图像检索系统的查询很少，所提出的攻击实现了较高的攻击成功率。实际视觉搜索引擎上的攻击评估表明，它成功欺骗了商业系统，例如Bing Visual Search，平均仅33个查询，其攻击成功率达到了98％。