论文标题
利用基于变压器的极端多标签文本分类中的本地和全局特征
Exploiting Local and Global Features in Transformer-based Extreme Multi-label Text Classification
论文作者
论文摘要
极端的多标签文本分类（XMTC）是用来自预定义类别的很大空间的相关标签标记每个文档的任务。最近，大型预训练的变压器模型已在XMTC中进行了重大的性能改进，该模型通常使用特殊CLS令牌的嵌入来代表整个文档语义作为全局功能向量，并与候选标签相匹配。但是，我们认为，这样的全球特征向量可能不足以表示文档中语义的不同粒度水平，并且与本地单词级别的功能相辅相成可能会带来更多的收益。基于这种见解，我们提出了一种结合变压器模型生成的本地和全局特征以提高分类器的预测能力的方法。我们的实验表明，所提出的模型要么胜过表现，要么与基准数据集上的最新方法相媲美。
Extreme multi-label text classification (XMTC) is the task of tagging each document with the relevant labels from a very large space of predefined categories. Recently, large pre-trained Transformer models have made significant performance improvements in XMTC, which typically use the embedding of the special CLS token to represent the entire document semantics as a global feature vector, and match it against candidate labels. However, we argue that such a global feature vector may not be sufficient to represent different granularity levels of semantics in the document, and that complementing it with the local word-level features could bring additional gains. Based on this insight, we propose an approach that combines both the local and global features produced by Transformer models to improve the prediction power of the classifier. Our experiments show that the proposed model either outperforms or is comparable to the state-of-the-art methods on benchmark datasets.
