论文标题
循环中的语言模型：将提示纳入弱监督
Language Models in the Loop: Incorporating Prompting into Weak Supervision
论文作者
论文摘要
我们提出了一种新的策略，即当标记的培训数据受到限制时，将大型预训练的语言模型应用于新任务。我们将模型视为在弱监督框架中标记功能的基础，而不是以典型的零拍或几次射击方式应用模型。为了创建分类器，我们首先提示该模型回答有关示例的多个不同的查询，并定义应如何将可能的响应映射到标签和弃权的投票。然后，我们使用浮潜系统将这些嘈杂的标签源定义，并通过最终分类器训练最终的培训数据。我们的实验评估表明，在弱监督框架内提示大型语言模型可以为准确性带来可观的提高。在扳手较弱的监督基准下，这种方法可以显着改善零击性能，平均误差减少19.5％。我们还发现，这种方法可产生与手工工程规则训练的分类器相当或卓越的精度。
We propose a new strategy for applying large pre-trained language models to novel tasks when labeled training data is limited. Rather than apply the model in a typical zero-shot or few-shot fashion, we treat the model as the basis for labeling functions in a weak supervision framework. To create a classifier, we first prompt the model to answer multiple distinct queries about an example and define how the possible responses should be mapped to votes for labels and abstentions. We then denoise these noisy label sources using the Snorkel system and train an end classifier with the resulting training data. Our experimental evaluation shows that prompting large language models within a weak supervision framework can provide significant gains in accuracy. On the WRENCH weak supervision benchmark, this approach can significantly improve over zero-shot performance, an average 19.5% reduction in errors. We also find that this approach produces classifiers with comparable or superior accuracy to those trained from hand-engineered rules.
