论文标题
将上下文对向量进行图形翻新以改进单词嵌入
Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings
论文作者
论文摘要
尽管由大规模预训练模型产生的上下文化嵌入在许多任务中都表现良好，但传统的静态嵌入（例如，跳过，word2vec）仍然在低资源和轻量级设置中由于其低计算成本，可轻松的部署和稳定性而发挥着重要作用。在本文中，我们旨在通过1）将现有预训练模型中的更多上下文信息纳入跳过框架中，我们将其称为“上下文到VEC”； 2）通过使用先验同义词知识和加权向量分布，提出一种用于静态嵌入的后处理改造方法。通过外在和内在任务，我们的方法被很好地证明可以超过基准。
Although contextualized embeddings generated from large-scale pre-trained models perform well in many tasks, traditional static embeddings (e.g., Skip-gram, Word2Vec) still play an important role in low-resource and lightweight settings due to their low computational cost, ease of deployment, and stability. In this paper, we aim to improve word embeddings by 1) incorporating more contextual information from existing pre-trained models into the Skip-gram framework, which we call Context-to-Vec; 2) proposing a post-processing retrofitting method for static embeddings independent of training by employing priori synonym knowledge and weighted vector distribution. Through extrinsic and intrinsic tasks, our methods are well proven to outperform the baselines by a large margin.
