论文标题
让模型决定其多任务学习课程
Let the Model Decide its Curriculum for Multitask Learning
论文作者
论文摘要
以前的多任务学习方法中的课程学习策略是基于人类感知或详尽搜索最佳安排的困难层次结构的数据集。但是，人类对难度的看法可能并不总是与机器解释良好相关，导致性能差，而详尽的搜索在计算上很昂贵。在解决这些问题时，我们提出了两类技术，以基于通过基于模型的方法计算的难度分数将培训实例安排为学习课程。这两个类别（即数据集级别和实例级别的排列粒度）不同。通过对12个数据集进行的全面实验，我们表明实例级别和数据集级别的技术导致了强大的表示形式，因为它们的平均性能提高了4.17％，比各自的基线的平均性能提高了3.15％。此外，我们发现大多数改进来自正确回答困难的实例，这意味着我们对艰巨任务的技术有更大的功效。
Curriculum learning strategies in prior multi-task learning approaches arrange datasets in a difficulty hierarchy either based on human perception or by exhaustively searching the optimal arrangement. However, human perception of difficulty may not always correlate well with machine interpretation leading to poor performance and exhaustive search is computationally expensive. Addressing these concerns, we propose two classes of techniques to arrange training instances into a learning curriculum based on difficulty scores computed via model-based approaches. The two classes i.e Dataset-level and Instance-level differ in granularity of arrangement. Through comprehensive experiments with 12 datasets, we show that instance-level and dataset-level techniques result in strong representations as they lead to an average performance improvement of 4.17% and 3.15% over their respective baselines. Furthermore, we find that most of this improvement comes from correctly answering the difficult instances, implying a greater efficacy of our techniques on difficult tasks.
