论文标题
使用以任务为导向的对话系统的选项框架对话策略与自然语言生成器之间的层次结构建模
Modelling Hierarchical Structure between Dialogue Policy and Natural Language Generator with Option Framework for Task-oriented Dialogue System
论文作者
论文摘要
设计面向任务的对话系统是一个具有挑战性的研究主题，因为它不仅需要生成满足用户请求的话语，而且还需要确保可理解性。许多以前的作品都用监督学习（SL）培训了端到端（E2E）模型，但是，带注释的系统话语的偏见仍然是一种瓶颈。强化学习（RL）通过使用非差异性评估指标（例如成功率）作为奖励来处理问题。尽管如此，与RL的现有作品表明，在提高满足用户请求的性能时，生成的系统话语的可理解性可能会破坏。在我们的工作中，我们（1）建议使用称为HDNO的选项框架对话策略与自然语言生成器（NLG）之间的层次结构进行建模，其中使用了《潜在对话法》来避免设计特定的对话ACT表示。 （2）通过层次增强学习（HRL）训练HDNO，并在培训期间提出对话政策与NLG之间的异步更新，从理论上保证它们与本地最大化器的融合； （3）建议使用以语言模型建模的歧视者作为进一步提高可理解性的额外奖励。我们在多域对话的数据集中测试了Multiwoz 2.0和Multiwoz 2.1的HDNO，与用RL，LARL和HDSA培训的单词级E2E模型相比，我们测试了多域对话的数据集，显示了通过自动评估指标和人类评估评估的性能的改进。最后，我们演示了潜在对话的语义含义，以展示HDNO的解释性。
Designing task-oriented dialogue systems is a challenging research topic, since it needs not only to generate utterances fulfilling user requests but also to guarantee the comprehensibility. Many previous works trained end-to-end (E2E) models with supervised learning (SL), however, the bias in annotated system utterances remains as a bottleneck. Reinforcement learning (RL) deals with the problem through using non-differentiable evaluation metrics (e.g., the success rate) as rewards. Nonetheless, existing works with RL showed that the comprehensibility of generated system utterances could be corrupted when improving the performance on fulfilling user requests. In our work, we (1) propose modelling the hierarchical structure between dialogue policy and natural language generator (NLG) with the option framework, called HDNO, where the latent dialogue act is applied to avoid designing specific dialogue act representations; (2) train HDNO via hierarchical reinforcement learning (HRL), as well as suggest the asynchronous updates between dialogue policy and NLG during training to theoretically guarantee their convergence to a local maximizer; and (3) propose using a discriminator modelled with language models as an additional reward to further improve the comprehensibility. We test HDNO on MultiWoz 2.0 and MultiWoz 2.1, the datasets on multi-domain dialogues, in comparison with word-level E2E model trained with RL, LaRL and HDSA, showing improvements on the performance evaluated by automatic evaluation metrics and human evaluation. Finally, we demonstrate the semantic meanings of latent dialogue acts to show the explanability for HDNO.
