论文标题
TREC演员2019：对话援助轨道概述
TREC CAsT 2019: The Conversational Assistance Track Overview
论文作者
论文摘要
对话援助轨道（CAST）是TREC 2019的新轨道，旨在促进对话信息寻求（CIS）研究，并为对话搜索系统创建大规模可重复使用的测试收集。文档语料库是TREC复杂答案检索（CAR）和Microsoft Machine Reading Gractension（Marco）数据集的38,426,252段落。寻求对话的80个信息（30列火车，50次测试）平均为9至10个问题。提供30个培训主题和20个测试主题的相关性评估。今年，有21个小组使用不同的方法提交了65次运行，以进行对话查询理解和排名。方法包括基于传统检索的方法，基于特征的学习对象，神经模型和知识增强方法。运行中的一个共同主题是使用基于BERT的神经reranking方法。领先的方法还采用了文档扩展，对话查询扩展和对话查询重写（GPT-2）的生成语言模型。结果表明，自动系统与使用手动解决的话语的系统之间存在差距，手动重写比最佳自动系统相对改进了35％。
The Conversational Assistance Track (CAsT) is a new track for TREC 2019 to facilitate Conversational Information Seeking (CIS) research and to create a large-scale reusable test collection for conversational search systems. The document corpus is 38,426,252 passages from the TREC Complex Answer Retrieval (CAR) and Microsoft MAchine Reading COmprehension (MARCO) datasets. Eighty information seeking dialogues (30 train, 50 test) are an average of 9 to 10 questions long. Relevance assessments are provided for 30 training topics and 20 test topics. This year 21 groups submitted a total of 65 runs using varying methods for conversational query understanding and ranking. Methods include traditional retrieval based methods, feature based learning-to-rank, neural models, and knowledge enhanced methods. A common theme through the runs is the use of BERT-based neural reranking methods. Leading methods also employed document expansion, conversational query expansion, and generative language models for conversational query rewriting (GPT-2). The results show a gap between automatic systems and those using the manually resolved utterances, with a 35% relative improvement of manual rewrites over the best automatic system.
