论文标题
关于图形卷积网络的全球自我注意机制
On the Global Self-attention Mechanism for Graph Convolutional Networks
论文作者
论文摘要
在特征上应用全球自我注意力（GSA）机制已在卷积神经网络（CNN）上取得了巨大的成功。但是，尚不清楚图形卷积网络（GCN）是否可以从这种技术中受益。在本文中，受到CNNS和GCN之间相似性的启发，我们研究了全球自我注意机制对GCN的影响。我们发现，与直觉一致，GSA机制允许GCN捕获基于特征的顶点关系，而不论边缘连接如何。结果，GSA机制可以为GCN引入额外的表达能力。此外，我们分析了GSA机制对过度拟合和过度光滑的问题的影响。我们证明，GSA机制可以根据一些最近的技术发展来减轻过度拟合和过度平滑的问题。在多个基准数据集上进行的实验既说明了出色的表达能力，又说明了GSA增强的GCN的出色表达能力和过度平滑的问题，这证实了直觉和理论结果。
Applying Global Self-attention (GSA) mechanism over features has achieved remarkable success on Convolutional Neural Networks (CNNs). However, it is not clear if Graph Convolutional Networks (GCNs) can similarly benefit from such a technique. In this paper, inspired by the similarity between CNNs and GCNs, we study the impact of the Global Self-attention mechanism on GCNs. We find that consistent with the intuition, the GSA mechanism allows GCNs to capture feature-based vertex relations regardless of edge connections; As a result, the GSA mechanism can introduce extra expressive power to the GCNs. Furthermore, we analyze the impacts of the GSA mechanism on the issues of overfitting and over-smoothing. We prove that the GSA mechanism can alleviate both the overfitting and the over-smoothing issues based on some recent technical developments. Experiments on multiple benchmark datasets illustrate both superior expressive power and less significant overfitting and over-smoothing problems for the GSA-augmented GCNs, which corroborate the intuitions and the theoretical results.
