论文标题
语法替代性无监督依赖性语法
Syntactic Substitutability as Unsupervised Dependency Syntax
论文作者
论文摘要
语法是一种潜在的层次结构，它是人类语言强大和构图的基础。在这项工作中，我们探讨了以下假设：语言模型的注意分布可以表示句法依赖性，并提出了一种新方法来诱导这些结构 - 迫切地诱导这些结构。我们没有建模注释模式定义的句法关系，而是对依赖关系（句法替代性的定义）的更一般属性进行建模。该属性捕获了以下事实：依赖关系的任何一端的单词都可以用同一类别的单词代替。替换可用于生成一组句法不变的句子，然后将其表示为解析。我们表明，增加所使用的替换数量提高了自然数据的解析精度。关于长距离主体 - 动词协议的结构，我们的方法使用以前的方法获得了79.5％的召回率，而8.9％的召回率为8.9％。我们的方法还将转移到不同的解析设置时提供改进，证明其概括。
Syntax is a latent hierarchical structure which underpins the robust and compositional nature of human language. In this work, we explore the hypothesis that syntactic dependencies can be represented in language model attention distributions and propose a new method to induce these structures theory-agnostically. Instead of modeling syntactic relations as defined by annotation schemata, we model a more general property implicit in the definition of dependency relations, syntactic substitutability. This property captures the fact that words at either end of a dependency can be substituted with words from the same category. Substitutions can be used to generate a set of syntactically invariant sentences whose representations are then used for parsing. We show that increasing the number of substitutions used improves parsing accuracy on natural data. On long-distance subject-verb agreement constructions, our method achieves 79.5% recall compared to 8.9% using a previous method. Our method also provides improvements when transferred to a different parsing setup, demonstrating that it generalizes.
