论文标题
伟大的真理总是很简单：一种相当简单的知识编码器，用于增强预训练模型的常识性推理能力
Great Truths are Always Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models
论文作者
论文摘要
自然语言中的常识推理是人工智能系统的所需能力。为了解决复杂的常识性推理任务，一种典型的解决方案是使用知识吸引的图形神经网络〜（GNN）编码器增强预训练的语言模型〜（PTMS），该编码模拟常识性知识图〜（CSKG）。尽管有效，这些方法还是建立在繁重的建筑上，无法清楚地解释外部知识资源如何提高PTM的推理能力。考虑到这个问题，我们进行了深入的经验分析，发现确实是CSKGS（但不是节点特征）的关系特征主要有助于PTM的性能提高。基于这一发现，我们设计了一个简单的基于MLP的知识编码器，该编码器利用统计关系路径作为特征。在五个基准上进行的广泛实验证明了我们方法的有效性，这也大大降低了编码CSKG的参数。我们的代码和数据可在https://github.com/rucaibox/safe上公开获取。
Commonsense reasoning in natural language is a desired ability of artificial intelligent systems. For solving complex commonsense reasoning tasks, a typical solution is to enhance pre-trained language models~(PTMs) with a knowledge-aware graph neural network~(GNN) encoder that models a commonsense knowledge graph~(CSKG). Despite the effectiveness, these approaches are built on heavy architectures, and can't clearly explain how external knowledge resources improve the reasoning capacity of PTMs. Considering this issue, we conduct a deep empirical analysis, and find that it is indeed relation features from CSKGs (but not node features) that mainly contribute to the performance improvement of PTMs. Based on this finding, we design a simple MLP-based knowledge encoder that utilizes statistical relation paths as features. Extensive experiments conducted on five benchmarks demonstrate the effectiveness of our approach, which also largely reduces the parameters for encoding CSKGs. Our codes and data are publicly available at https://github.com/RUCAIBox/SAFE.
