论文标题
在BERT中的常识线索上解决常识任务
On Commonsense Cues in BERT for Solving Commonsense Tasks
论文作者
论文摘要
Bert已用于解决常识任务，例如CommonSenseQA。尽管先前的研究发现BERT确实包含常识性信息，但有工作表明，预训练的模型可以依靠虚假的关联（例如数据偏见），而不是解决情感分类和其他问题方面的关键提示。在解决常识任务时，我们定量地研究了BERT中结构常识线索的存在，以及此类提示对于模型预测的重要性。使用两种不同的措施，我们发现伯特确实使用相关知识来解决任务，而常识知识的存在与模型的准确性正相关。
BERT has been used for solving commonsense tasks such as CommonsenseQA. While prior research has found that BERT does contain commonsense information to some extent, there has been work showing that pre-trained models can rely on spurious associations (e.g., data bias) rather than key cues in solving sentiment classification and other problems. We quantitatively investigate the presence of structural commonsense cues in BERT when solving commonsense tasks, and the importance of such cues for the model prediction. Using two different measures, we find that BERT does use relevant knowledge for solving the task, and the presence of commonsense knowledge is positively correlated to the model accuracy.
