论文标题
命名实体识别的粗到精细预训练
Coarse-to-Fine Pre-training for Named Entity Recognition
论文作者
论文摘要
最近，命名的实体识别可以在伯特（Bert）等培训前的帮助下进行了巨大的进步。但是，当前的训练技术着重于构建Lan-Guage建模目标，以学习Gen-orser代表，而忽略了与命名的实体相关的知识。为此，我们建议特定于特定的预训练框架，以自动挖掘到预先训练的模型中，以自动挖掘的实体知识。特殊情况下，我们首先通过对Wikipedia锚进行训练，首先通过辅助跨度识别任务进行预热，这可以视为被视为ASERALTY型实体。然后，我们利用基于Gazetteer的远处监督策略进行模型提取的粗粒粒度打字。最后，我们设计了一项自我讨论的任务，以通过聚类来挖掘细粒度的名称知识。对三个公共NER数据集的经验研究恶魔策略，我们的框架对几个预训练的基础线实现了重大的影响，从而确立了在三个基本标记上建立新的现有的现实状态。此外，我们的框架在不使用人体标记的训练数据Data的情况下获得了有希望的重新塑造，这表明了其在标签 - 面上和低资源场景中的有效性
More recently, Named Entity Recognition hasachieved great advances aided by pre-trainingapproaches such as BERT. However, currentpre-training techniques focus on building lan-guage modeling objectives to learn a gen-eral representation, ignoring the named entity-related knowledge. To this end, we proposea NER-specific pre-training framework to in-ject coarse-to-fine automatically mined entityknowledge into pre-trained models. Specifi-cally, we first warm-up the model via an en-tity span identification task by training it withWikipedia anchors, which can be deemed asgeneral-typed entities. Then we leverage thegazetteer-based distant supervision strategy totrain the model extract coarse-grained typedentities. Finally, we devise a self-supervisedauxiliary task to mine the fine-grained namedentity knowledge via clustering.Empiricalstudies on three public NER datasets demon-strate that our framework achieves significantimprovements against several pre-trained base-lines, establishing the new state-of-the-art per-formance on three benchmarks. Besides, weshow that our framework gains promising re-sults without using human-labeled trainingdata, demonstrating its effectiveness in label-few and low-resource scenarios
