论文标题
探索自我发作以识别图像
Exploring Self-attention for Image Recognition
论文作者
论文摘要
最近的工作表明，自我注意力可以作为图像识别模型的基本基础。我们探索自我注意力的变化，并评估其图像识别的有效性。我们考虑两种形式的自我注意力。一个是成对的自我注意力，它概括了标准的点产生关注，从根本上讲是设定的操作员。另一个是斑点自我注意力，严格比卷积更强大。我们的成对自我发场网络匹配或胜过其卷积对应物，而PatchWise模型基本上优于卷积基线。我们还进行了实验，以探测学习代表的鲁棒性，并得出结论，自我发项网络在鲁棒性和概括方面可能具有重大好处。
Recent work has shown that self-attention can serve as a basic building block for image recognition models. We explore variations of self-attention and assess their effectiveness for image recognition. We consider two forms of self-attention. One is pairwise self-attention, which generalizes standard dot-product attention and is fundamentally a set operator. The other is patchwise self-attention, which is strictly more powerful than convolution. Our pairwise self-attention networks match or outperform their convolutional counterparts, and the patchwise models substantially outperform the convolutional baselines. We also conduct experiments that probe the robustness of learned representations and conclude that self-attention networks may have significant benefits in terms of robustness and generalization.
