论文标题
学习分解：基于可比文本的假设问题分解
Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Texts
论文作者
论文摘要
显式分解建模涉及将复杂的任务分解为更直接，通常更容易解释的子任务，一直是开发坚固且可解释的NLU系统的核心主题。但是，尽管作为这项工作的一部分构建了许多数据集和资源，但大多数人具有小规模的注释和有限的范围，这不足以解决一般的分解任务。在本文中，我们使用可比文本（尤其是大规模平行新闻）的遥远监督来研究基于分解变压器的大规模中间训练。我们表明，随着这种中间的预训练，开发出可靠的分解模型，以实现各种任务范围的模型变得更加可行。例如，在语义解析上，我们的模型DeTompt5在两个数据集（过夜和扭矩）上，在基线语言模型上提高了20％至30％。我们进一步使用DECOMPT5来构建一个名为Depancentail的新型基于分解的QA系统，在HOTPOTQA和StrategyQA上分别提高了包括GPT-3在内的最先进模型，分别提高了8％和4％。
Explicit decomposition modeling, which involves breaking down complex tasks into more straightforward and often more interpretable sub-tasks, has long been a central theme in developing robust and interpretable NLU systems. However, despite the many datasets and resources built as part of this effort, the majority have small-scale annotations and limited scope, which is insufficient to solve general decomposition tasks. In this paper, we look at large-scale intermediate pre-training of decomposition-based transformers using distant supervision from comparable texts, particularly large-scale parallel news. We show that with such intermediate pre-training, developing robust decomposition-based models for a diverse range of tasks becomes more feasible. For example, on semantic parsing, our model, DecompT5, improves 20% to 30% on two datasets, Overnight and TORQUE, over the baseline language model. We further use DecompT5 to build a novel decomposition-based QA system named DecompEntail, improving over state-of-the-art models, including GPT-3, on both HotpotQA and StrategyQA by 8% and 4%, respectively.
