论文标题
学习标签模块化提示在野外进行文本分类
Learning Label Modular Prompts for Text Classification in the Wild
论文作者
论文摘要
机器学习模型通常在培训和测试期间假设I.I.D数据，但是现实世界中的数据和任务通常会随着时间而变化。为了模仿现实世界的瞬时性质，我们提出了一项具有挑战性但实用的任务：内部文本分类，它引入了不同的非平稳培训/测试阶段。将复杂的任务分解为模块化组件可以在这种非平稳环境下实现强大的概括。但是，NLP中当前的模块化方法并不能利用审计语言模型的参数有效调整的最新进展。为了缩小此差距，我们提出了ModularPrompt，这是用于文本分类任务的标签模数提示调谐框架。在Modular Prompt中，输入提示包括一系列软标签提示，每个提示符都编码与相应类标签有关的模块化知识。在最强大的两个设置中，模块化的幅度优于相关的基线，其边缘表现出强大的概括能力。我们还进行了全面的分析，以验证学习的提示是否满足模块化表示的性能。
Machine learning models usually assume i.i.d data during training and testing, but data and tasks in real world often change over time. To emulate the transient nature of real world, we propose a challenging but practical task: text classification in-the-wild, which introduces different non-stationary training/testing stages. Decomposing a complex task into modular components can enable robust generalisation under such non-stationary environment. However, current modular approaches in NLP do not take advantage of recent advances in parameter efficient tuning of pretrained language models. To close this gap, we propose MODULARPROMPT, a label-modular prompt tuning framework for text classification tasks. In MODULARPROMPT, the input prompt consists of a sequence of soft label prompts, each encoding modular knowledge related to the corresponding class label. In two of most formidable settings, MODULARPROMPT outperforms relevant baselines by a large margin demonstrating strong generalisation ability. We also conduct comprehensive analysis to validate whether the learned prompts satisfy properties of a modular representation.
