论文标题
多任务模型对于结构性失败是可靠的：双语认知储备的神经模型
Multitasking Models are Robust to Structural Failure: A Neural Model for Bilingual Cognitive Reserve
论文作者
论文摘要
我们发现多任务学习与神经元失败之间的鲁棒性之间存在令人惊讶的联系。我们的实验表明，与等效单语言相比，双语语言模型在各种神经元扰动下保留了更高的性能，例如随机缺失，幅度修剪和重量噪声。我们通过数学分析线性表示学习并表明多任务会创造更强大的表示形式来为这种鲁棒性提供理论上的理由。我们的分析将鲁棒性与学习表示的光谱特性联系起来，并证明多任务导致更高的鲁棒性对于多样化的任务向量。我们开放代码和模型：https：//github.com/giannisdaras/multlingual_robustness
We find a surprising connection between multitask learning and robustness to neuron failures. Our experiments show that bilingual language models retain higher performance under various neuron perturbations, such as random deletions, magnitude pruning and weight noise compared to equivalent monolingual ones. We provide a theoretical justification for this robustness by mathematically analyzing linear representation learning and showing that multitasking creates more robust representations. Our analysis connects robustness to spectral properties of the learned representation and proves that multitasking leads to higher robustness for diverse task vectors. We open-source our code and models: https://github.com/giannisdaras/multilingual_robustness
