论文标题
提示语言结构的语言模型
Prompting Language Models for Linguistic Structure
论文作者
论文摘要
尽管可以提示验证的语言模型（PLM）执行各种语言任务，但仍然是一个悬而未决的问题，这种能力来自可概括的语言理解与表面级词汇模式。为了测试这一点，我们为语言结构化预测任务提供了一种结构化提示方法，使我们能够使用自回归PLMS执行零和少量序列标记。我们在言论的一部分标签，命名实体识别和句子块上评估了这种方法，在所有情况下都表明了很少的表现。我们还发现，虽然PLM由于任务泄漏到训练训练语料库而包含了重要的任务标签知识，但结构化提示也可以使用任意标签检索语言结构。这些发现表明，PLMS的文化学习能力和语言知识在其训练数据的记忆之外概括了。
Although pretrained language models (PLMs) can be prompted to perform a wide range of language tasks, it remains an open question how much this ability comes from generalizable linguistic understanding versus surface-level lexical patterns. To test this, we present a structured prompting approach for linguistic structured prediction tasks, allowing us to perform zero- and few-shot sequence tagging with autoregressive PLMs. We evaluate this approach on part-of-speech tagging, named entity recognition, and sentence chunking, demonstrating strong few-shot performance in all cases. We also find that while PLMs contain significant prior knowledge of task labels due to task leakage into the pretraining corpus, structured prompting can also retrieve linguistic structure with arbitrary labels. These findings indicate that the in-context learning ability and linguistic knowledge of PLMs generalizes beyond memorization of their training data.
