论文标题
命名实体识别的深度跨度表示
Deep Span Representations for Named Entity Recognition
论文作者
论文摘要
基于跨度的模型是指定实体识别（NER）最直接的方法之一。现有的基于跨度的NER系统浅水将令牌表示形式汇总为跨度表示。但是，这通常会导致长跨度实体，重叠跨度的表示之间的耦合以及最终的性能降解。在这项研究中，我们提出了DSPERT（Transformers的深层编码器表示），其中包括标准变压器和跨度变压器。后者将低层跨度表示作为查询，并将令牌表示为键和值，从底部到顶部逐层逐层。因此，dspert会产生深层语义的跨度表示。 通过预验证的语言模型的重量初始化，DSPERT的性能高于八个NER基准测试的最新最新系统。实验结果验证了深度对跨度表示的重要性，并表明DSPERT在长跨度实体和嵌套结构上表现良好。此外，深层表示结构良好，并且在特征空间中易于分离。
Span-based models are one of the most straightforward methods for named entity recognition (NER). Existing span-based NER systems shallowly aggregate the token representations to span representations. However, this typically results in significant ineffectiveness for long-span entities, a coupling between the representations of overlapping spans, and ultimately a performance degradation. In this study, we propose DSpERT (Deep Span Encoder Representations from Transformers), which comprises a standard Transformer and a span Transformer. The latter uses low-layered span representations as queries, and aggregates the token representations as keys and values, layer by layer from bottom to top. Thus, DSpERT produces span representations of deep semantics. With weight initialization from pretrained language models, DSpERT achieves performance higher than or competitive with recent state-of-the-art systems on eight NER benchmarks. Experimental results verify the importance of the depth for span representations, and show that DSpERT performs particularly well on long-span entities and nested structures. Further, the deep span representations are well structured and easily separable in the feature space.
