论文标题
稀疏和结构化的视觉注意
Sparse and Structured Visual Attention
论文作者
论文摘要
视觉注意机制被广泛用于多模式任务，作为视觉问题回答（VQA）。基于软马克斯的注意机制的一个缺点是，它们为所有图像区域分配了一些概率质量，无论其邻接结构以及与文本的相关性如何。 In this paper, to better link the image structure with the text, we replace the traditional softmax attention mechanism with two alternative sparsity-promoting transformations: sparsemax, which is able to select only the relevant regions (assigning zero weight to the rest), and a newly proposed Total-Variation Sparse Attention (TVmax), which further encourages the joint selection of adjacent spatial locations. VQA的实验表明，与人类注意力的准确性以及更高的相似性，这表明可以更好地解释性。
Visual attention mechanisms are widely used in multimodal tasks, as visual question answering (VQA). One drawback of softmax-based attention mechanisms is that they assign some probability mass to all image regions, regardless of their adjacency structure and of their relevance to the text. In this paper, to better link the image structure with the text, we replace the traditional softmax attention mechanism with two alternative sparsity-promoting transformations: sparsemax, which is able to select only the relevant regions (assigning zero weight to the rest), and a newly proposed Total-Variation Sparse Attention (TVmax), which further encourages the joint selection of adjacent spatial locations. Experiments in VQA show gains in accuracy as well as higher similarity to human attention, which suggests better interpretability.
