论文标题
通过分析特征归因来识别合适的归纳转移任务
Identifying Suitable Tasks for Inductive Transfer Through the Analysis of Feature Attributions
论文作者
论文摘要
转移学习方法已显示出可显着提高下游任务的性能。但是，先前的工作通常只报告转移学习是有益的，而忽略了寻找有效转移设置所需的重大试验和错误。实际上，并非所有的任务组合都会带来绩效益处，并且迅速搜索蛮力在计算上变得不可行。因此，问题出现了，我们可以预测两个任务之间的转移是否会在没有实际执行实验的情况下是有益的？在本文中，我们利用解释性技术有效地预测任务对是否会互补，通过比较单任务模型之间的神经网络激活。这样，我们可以避免在所有任务和超参数组合上进行网格搜索，从而大大减少找到有效任务对所需的时间。我们的结果表明，通过这种方法，可以将训练时间减少多达83.5％，而TREC-IS 2020-A数据集的正类F1的成本仅减少0.034。
Transfer learning approaches have shown to significantly improve performance on downstream tasks. However, it is common for prior works to only report where transfer learning was beneficial, ignoring the significant trial-and-error required to find effective settings for transfer. Indeed, not all task combinations lead to performance benefits, and brute-force searching rapidly becomes computationally infeasible. Hence the question arises, can we predict whether transfer between two tasks will be beneficial without actually performing the experiment? In this paper, we leverage explainability techniques to effectively predict whether task pairs will be complementary, through comparison of neural network activation between single-task models. In this way, we can avoid grid-searches over all task and hyperparameter combinations, dramatically reducing the time needed to find effective task pairs. Our results show that, through this approach, it is possible to reduce training time by up to 83.5% at a cost of only 0.034 reduction in positive-class F1 on the TREC-IS 2020-A dataset.
