论文标题
低资源多任务序列标签 - 重新访问动态条件随机字段
Low Resource Multi-Task Sequence Tagging -- Revisiting Dynamic Conditional Random Fields
论文作者
论文摘要
我们比较了低资源多任务序列标签的不同模型，即在标签序列之间用于不同任务的标签序列之间的依赖性。我们的分析针对的是每个示例都有用于多个任务的标签的数据集。当前方法使用每个任务的单独模型或标准的多任务学习来学习共享的特征表示。但是，这些方法忽略了标签序列之间的相关性，这些序列可以通过小型培训数据集在设置中提供重要信息。为了分析哪些方案可以从不同任务中标签之间的依赖关系中获利，我们重新审视动态条件随机字段（CRF），并将它们与深神经网络相结合。我们比较单个任务，多任务和动态CRF设置，用于在英语和德语低资源方案中的句子和文档级别的三个不同数据集。我们表明，包括概括性词性标签者的银标签，因为辅助任务可以改善下游任务的性能。我们发现，尤其是在低资源场景中，任务预测之间相互依存关系的显式建模优于单任务和标准的多任务模型。
We compare different models for low resource multi-task sequence tagging that leverage dependencies between label sequences for different tasks. Our analysis is aimed at datasets where each example has labels for multiple tasks. Current approaches use either a separate model for each task or standard multi-task learning to learn shared feature representations. However, these approaches ignore correlations between label sequences, which can provide important information in settings with small training datasets. To analyze which scenarios can profit from modeling dependencies between labels in different tasks, we revisit dynamic conditional random fields (CRFs) and combine them with deep neural networks. We compare single-task, multi-task and dynamic CRF setups for three diverse datasets at both sentence and document levels in English and German low resource scenarios. We show that including silver labels from pretrained part-of-speech taggers as auxiliary tasks can improve performance on downstream tasks. We find that especially in low-resource scenarios, the explicit modeling of inter-dependencies between task predictions outperforms single-task as well as standard multi-task models.
