论文标题
人类和语言模型中的附带促进
Collateral facilitation in humans and language models
论文作者
论文摘要
人类和语言模型的预测是否受类似事物影响？研究表明，在理解语言的同时，人类对即将到来的单词做出了预测，更容易地处理更可预测的单词。但是，有证据表明，当这些单词与前面的上下文或与最可能的延续相关时，人类对高度异常单词的处理优势显示出类似的处理优势。使用3个心理语言实验的刺激，我们发现8种当代变压器语言模型（Bert，Albert，Roberta，XLM-R，GPT-2，GPT-2，GPT-NEO，GPT-NEO，GPT-J和XGLM）几乎总是如此。然后，我们讨论这种现象对我们对人类语言理解的理解和语言模型的预测的含义。
Are the predictions of humans and language models affected by similar things? Research suggests that while comprehending language, humans make predictions about upcoming words, with more predictable words being processed more easily. However, evidence also shows that humans display a similar processing advantage for highly anomalous words when these words are semantically related to the preceding context or to the most probable continuation. Using stimuli from 3 psycholinguistic experiments, we find that this is also almost always also the case for 8 contemporary transformer language models (BERT, ALBERT, RoBERTa, XLM-R, GPT-2, GPT-Neo, GPT-J, and XGLM). We then discuss the implications of this phenomenon for our understanding of both human language comprehension and the predictions made by language models.
