论文标题
话语结构与神经语言模型中的参考文献相互作用，但不是语法
Discourse structure interacts with reference but not syntax in neural language models
论文作者
论文摘要
已经据称接受大量文本培训的语言模型（LMS）获得了抽象的语言表示。我们的工作通过着重于LMS学习不同语言表示之间的相互作用的能力来测试这些抽象的鲁棒性。特别是，我们从心理语言学研究中利用刺激，表明人可以在相同的话语结构（隐式因果关系）上调节参考（即核心分辨率）和句法处理。我们比较了变形金刚和长期的短期记忆LMS，以发现与人类相反，隐式因果关系仅影响参考的LM行为，而不是语法，尽管模型表示编码了必要的话语信息。我们的结果进一步表明，LM行为不仅可以矛盾，不仅可以矛盾话语的表示，而且还与句法一致性相矛盾，这表明标准语言建模的缺点。
Language models (LMs) trained on large quantities of text have been claimed to acquire abstract linguistic representations. Our work tests the robustness of these abstractions by focusing on the ability of LMs to learn interactions between different linguistic representations. In particular, we utilized stimuli from psycholinguistic studies showing that humans can condition reference (i.e. coreference resolution) and syntactic processing on the same discourse structure (implicit causality). We compared both transformer and long short-term memory LMs to find that, contrary to humans, implicit causality only influences LM behavior for reference, not syntax, despite model representations that encode the necessary discourse information. Our results further suggest that LM behavior can contradict not only learned representations of discourse but also syntactic agreement, pointing to shortcomings of standard language modeling.
