论文标题
rvae-lamol：剩余的变异自动编码器，以增强终身语言学习
RVAE-LAMOL: Residual Variational Autoencoder to Enhance Lifelong Language Learning
论文作者
论文摘要
终身语言学习（LLL）旨在训练神经网络以学习NLP任务流，同时保留以前的任务知识。但是，遵循无数据约束的先前作品仍然遭受灾难性遗忘问题的困扰，该模型忘记了从以前的任务中学到的东西。为了减轻灾难性的遗忘，我们建议通过将不同的任务映射到有限的统一语义空间中，以增强剩余的变异自动编码器（RVAE）来增强Lamol（最近的LLL模型）。在这个空间中，以前的任务很容易通过伪样本正确正确。此外，我们提出了一个身份任务，以使模型具有歧视性，以识别属于哪个任务的样本。为了更好地训练RVAE-Lamol，我们建议一种新颖的训练计划替代滞后训练。在实验中，我们测试了decanlp的三个数据集排列的rvae-lamol。实验结果表明，RVAE-Lamol在所有排列上都胜过幼稚的lamol，并生成更有意义的伪示例。
Lifelong Language Learning (LLL) aims to train a neural network to learn a stream of NLP tasks while retaining knowledge from previous tasks. However, previous works which followed data-free constraint still suffer from catastrophic forgetting issue, where the model forgets what it just learned from previous tasks. In order to alleviate catastrophic forgetting, we propose the residual variational autoencoder (RVAE) to enhance LAMOL, a recent LLL model, by mapping different tasks into a limited unified semantic space. In this space, previous tasks are easy to be correct to their own distribution by pseudo samples. Furthermore, we propose an identity task to make the model is discriminative to recognize the sample belonging to which task. For training RVAE-LAMOL better, we propose a novel training scheme Alternate Lag Training. In the experiments, we test RVAE-LAMOL on permutations of three datasets from DecaNLP. The experimental results demonstrate that RVAE-LAMOL outperforms naïve LAMOL on all permutations and generates more meaningful pseudo-samples.
