论文标题
基于令牌的参数单位识别中测试鲁棒性的扰动和亚群
Perturbations and Subpopulations for Testing Robustness in Token-Based Argument Unit Recognition
论文作者
论文摘要
参数单位识别和分类旨在从文本中识别参数单元，并将其分类为Pro或反对。为此任务开发系统时需要做出的设计选择之一是分类单位应该是：令牌或完整句子的段。先前的研究表明，与直接对句子进行培训相比，对令牌级别的微调语言模型可用于对句子进行分类的更强大的结果。我们重现了最初提出这一主张的研究，并进一步研究了与基于句子的系统相比，基于代币的系统学会更好。我们开发了系统的测试，以分析基于令牌的系统和基于句子的系统之间的行为差异。我们的结果表明，基于令牌的模型通常比手动扰动的示例和数据的特定亚群都比基于句子的模型更强大。
Argument Unit Recognition and Classification aims at identifying argument units from text and classifying them as pro or against. One of the design choices that need to be made when developing systems for this task is what the unit of classification should be: segments of tokens or full sentences. Previous research suggests that fine-tuning language models on the token-level yields more robust results for classifying sentences compared to training on sentences directly. We reproduce the study that originally made this claim and further investigate what exactly token-based systems learned better compared to sentence-based ones. We develop systematic tests for analysing the behavioural differences between the token-based and the sentence-based system. Our results show that token-based models are generally more robust than sentence-based models both on manually perturbed examples and on specific subpopulations of the data.
