论文标题

向下和跨越:引入填字游戏作为新的NLP基准

Down and Across: Introducing Crossword-Solving as a New NLP Benchmark

论文作者

Kulshreshtha, Saurabh, Kovaleva, Olga, Shivagunde, Namrata, Rumshisky, Anna

论文摘要

解决填字游戏需要各种推理能力,获得有关语言和世界的大量知识,以及满足拼图结构所施加的约束的能力。在这项工作中,我们将解决填字游戏介绍为新的自然语言理解任务。我们发布了从《纽约时报》每日填字游戏中收集的填字游戏的规范,涵盖了25年,其中包括大约9000个难题。这些难题包括各种线索:历史,事实,单词含义,同义词/反义词,填充,缩写,前缀/后缀,文字播放和跨语言,以及取决于其他线索答案的线索。我们从这些难题中分别释放了线索 - 答案对,作为一个开放域的问题,回答包含超过半百万个独特的线索 - 答案对的数据集。对于问题回答任务,我们的基准包括几个顺序到序列和基于检索的生成模型。我们还引入了一个非参数约束满意度基线,以解决整个填字游戏。最后,我们提出了一个评估框架,该框架由几个互补的绩效指标组成。

Solving crossword puzzles requires diverse reasoning capabilities, access to a vast amount of knowledge about language and the world, and the ability to satisfy the constraints imposed by the structure of the puzzle. In this work, we introduce solving crossword puzzles as a new natural language understanding task. We release the specification of a corpus of crossword puzzles collected from the New York Times daily crossword spanning 25 years and comprised of a total of around nine thousand puzzles. These puzzles include a diverse set of clues: historic, factual, word meaning, synonyms/antonyms, fill-in-the-blank, abbreviations, prefixes/suffixes, wordplay, and cross-lingual, as well as clues that depend on the answers to other clues. We separately release the clue-answer pairs from these puzzles as an open-domain question answering dataset containing over half a million unique clue-answer pairs. For the question answering task, our baselines include several sequence-to-sequence and retrieval-based generative models. We also introduce a non-parametric constraint satisfaction baseline for solving the entire crossword puzzle. Finally, we propose an evaluation framework which consists of several complementary performance metrics.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源