论文标题
通过最大利用几乎没有反事实的鲁棒性情感分类
Robustifying Sentiment Classification by Maximally Exploiting Few Counterfactuals
论文作者
论文摘要
对于文本分类任务，填充语言模型的性能非常出色。但是，他们倾向于依靠训练数据中的虚假模式，从而将其性能限制在分布（OOD）测试数据上。在旨在避免这种虚假模式问题的最新模型中，在培训数据中添加额外的反事实样本已被证明非常有效。然而，反事实数据的生成是昂贵的，因为它依赖于人类注释。因此，我们提出了一个新的解决方案，该解决方案仅需要对原始训练数据的一小部分（例如1％）注释，并在编码矢量空间中使用自动生成额外的反事实。我们使用IMDB数据进行培训和其他用于OOD测试的集合（即亚马逊，Semeval和Yelp），证明了方法在情感分类中的有效性。通过仅添加1％的手动反事实来，我们获得了明显的准确性提高：与添加 +100％分布培训样本相比， +3％，与替代反事实方法相比， +1.3％。
For text classification tasks, finetuned language models perform remarkably well. Yet, they tend to rely on spurious patterns in training data, thus limiting their performance on out-of-distribution (OOD) test data. Among recent models aiming to avoid this spurious pattern problem, adding extra counterfactual samples to the training data has proven to be very effective. Yet, counterfactual data generation is costly since it relies on human annotation. Thus, we propose a novel solution that only requires annotation of a small fraction (e.g., 1%) of the original training data, and uses automatic generation of extra counterfactuals in an encoding vector space. We demonstrate the effectiveness of our approach in sentiment classification, using IMDb data for training and other sets for OOD tests (i.e., Amazon, SemEval and Yelp). We achieve noticeable accuracy improvements by adding only 1% manual counterfactuals: +3% compared to adding +100% in-distribution training samples, +1.3% compared to alternate counterfactual approaches.
