论文标题
从预训练的语言模型中探索明式知识
Probing Simile Knowledge from Pre-trained Language Models
论文作者
论文摘要
对于NLP来说,明喻解释(SI)和明喻生成(SG)是具有挑战性的任务,因为模型需要足够的世界知识来产生预测。以前的作品已经采用了许多手工制作的资源来将与知识相关的模型带入模型,这是耗时且劳动力密集的。近年来,基于预培训的语言模型(PLM)方法已成为NLP的事实上的标准,因为它们从大型语料库中学习了通用知识。 PLM中嵌入的知识可能对SI和SG任务有用。尽管如此,很少有作品可以探索它。在本文中,我们探究了PLMS的明喻知识,以在第一次完成Simile Triple完成的统一框架中求解SI和SG任务。我们框架的骨干是用手动模式构造蒙面句子,然后在蒙面位置预测候选单词。在此框架中,我们采用了二级培训过程(形容词掩码训练),并使用蒙版语言模型(MLM)损失,以增强掩盖位置中候选单词的预测多样性。此外,应用模式集合(PE)和模式搜索(PS)以提高预测单词的质量。最后,自动和人类评估证明了我们在SI和SG任务中框架的有效性。
Simile interpretation (SI) and simile generation (SG) are challenging tasks for NLP because models require adequate world knowledge to produce predictions. Previous works have employed many hand-crafted resources to bring knowledge-related into models, which is time-consuming and labor-intensive. In recent years, pre-trained language models (PLMs) based approaches have become the de-facto standard in NLP since they learn generic knowledge from a large corpus. The knowledge embedded in PLMs may be useful for SI and SG tasks. Nevertheless, there are few works to explore it. In this paper, we probe simile knowledge from PLMs to solve the SI and SG tasks in the unified framework of simile triple completion for the first time. The backbone of our framework is to construct masked sentences with manual patterns and then predict the candidate words in the masked position. In this framework, we adopt a secondary training process (Adjective-Noun mask Training) with the masked language model (MLM) loss to enhance the prediction diversity of candidate words in the masked position. Moreover, pattern ensemble (PE) and pattern search (PS) are applied to improve the quality of predicted words. Finally, automatic and human evaluations demonstrate the effectiveness of our framework in both SI and SG tasks.