论文标题
稳健多语言学习的参数效率填充
Parameter-Efficient Finetuning for Robust Continual Multilingual Learning
论文作者
论文摘要
我们介绍并研究了连续多语言学习（CML）的问题，其中使用新的数据阶段定期更新了先前训练的多语言模型。如果新数据仅以语言为一部分，我们发现所产生的模型仅在最新更新中包含的语言（以及几种密切相关的语言）中显示出改善的性能，而其剩余语言的性能显着降低。我们通过提出Laft-uriel（一种参数效率高效的填充策略）来应对这一挑战，该策略旨在增加模型在更新后改善的语言数量，同时减少其余语言的性能损失幅度。 LAFT-URIEL使用语言知识来平衡跨语言的过度拟合和知识共享，从而允许另外25％的任务语言看到更新后的性能提高，同时还将其余语言的平均损失幅度减少了78％。
We introduce and study the problem of Continual Multilingual Learning (CML) where a previously trained multilingual model is periodically updated using new data arriving in stages. If the new data is present only in a subset of languages, we find that the resulting model shows improved performance only on the languages included in the latest update (and a few closely related languages) while its performance on all the remaining languages degrade significantly. We address this challenge by proposing LAFT-URIEL, a parameter-efficient finetuning strategy which aims to increase the number of languages on which the model improves after an update, while reducing the magnitude of loss in performance for the remaining languages. LAFT-URIEL uses linguistic knowledge to balance overfitting and knowledge sharing across languages, allowing for an additional 25% of task languages to see an improvement in performance after an update, while also reducing the average magnitude of losses on the remaining languages by 78% relative.
