论文标题
线索：使用自然语言解释学习分类器的基准
CLUES: A Benchmark for Learning Classifiers using Natural Language Explanations
论文作者
论文摘要
传统上，有监督的学习通过观察标记的任务示例来关注归纳学习。相反，人类有能力从语言中学习新概念。在这里，我们仅从语言中探索零击分类器的结构化数据。为此，我们介绍了线索，这是使用自然语言解释的分类器学习的基准，其中包括一系列分类任务，包括结构化数据的一系列分类任务以及以解释形式的自然语言监督。线索由36个现实世界和144个合成分类任务组成。它包含众包解释，描述了来自多个教师的现实世界任务以及对合成任务的编程生成的解释。为了模拟解释在分类示例中的影响，我们开发了一个基于元素的模型，该模型使用解释来学习分类器。与不使用解释的基线相比，Exent对新任务的概括（相对）高达18％（相对）。我们从解释中解释了自动化学习的关键挑战，这可能会导致未来的线索进步。代码和数据集可在以下网址提供：https：//clues-benchmark.github.io。
Supervised learning has traditionally focused on inductive learning by observing labeled examples of a task. In contrast, humans have the ability to learn new concepts from language. Here, we explore training zero-shot classifiers for structured data purely from language. For this, we introduce CLUES, a benchmark for Classifier Learning Using natural language ExplanationS, consisting of a range of classification tasks over structured data along with natural language supervision in the form of explanations. CLUES consists of 36 real-world and 144 synthetic classification tasks. It contains crowdsourced explanations describing real-world tasks from multiple teachers and programmatically generated explanations for the synthetic tasks. To model the influence of explanations in classifying an example, we develop ExEnt, an entailment-based model that learns classifiers using explanations. ExEnt generalizes up to 18% better (relative) on novel tasks than a baseline that does not use explanations. We delineate key challenges for automated learning from explanations, addressing which can lead to progress on CLUES in the future. Code and datasets are available at: https://clues-benchmark.github.io.
