论文标题
请给我您的注意力：点产品的注意力被认为对对抗性贴片有害鲁棒性有害
Give Me Your Attention: Dot-Product Attention Considered Harmful for Adversarial Patch Robustness
论文作者
论文摘要
基于视觉变压器等注意力的神经体系结构正在彻底改变图像识别。他们的主要好处是，注意力允许共同推理场景的所有部分。在本文中，我们展示了（缩放）点产生关注的全球推理在面对对抗斑块攻击时可能成为主要脆弱性的根源。我们提供了对这种脆弱性的理论理解，并将其与对手在对抗性贴片控制下误导所有查询的注意力指向单个密钥令牌的能力。我们提出了新的对抗性目标，用于制定针对此脆弱性的对抗斑块。我们显示了对流行图像分类（VIT和DEIT）和对象检测模型（DETR）的拟议补丁攻击的有效性。我们发现，占据0.5％输入的对抗斑块可导致Imagenet上VIT的稳健精度低至0％，并将MS Coco上DETR的地图降低到小于3％。
Neural architectures based on attention such as vision transformers are revolutionizing image recognition. Their main benefit is that attention allows reasoning about all parts of a scene jointly. In this paper, we show how the global reasoning of (scaled) dot-product attention can be the source of a major vulnerability when confronted with adversarial patch attacks. We provide a theoretical understanding of this vulnerability and relate it to an adversary's ability to misdirect the attention of all queries to a single key token under the control of the adversarial patch. We propose novel adversarial objectives for crafting adversarial patches which target this vulnerability explicitly. We show the effectiveness of the proposed patch attacks on popular image classification (ViTs and DeiTs) and object detection models (DETR). We find that adversarial patches occupying 0.5% of the input can lead to robust accuracies as low as 0% for ViT on ImageNet, and reduce the mAP of DETR on MS COCO to less than 3%.
