论文标题
一种用于通过预训练的深神经网络分类器检测对话系统未知意图的后处理方法
A Post-processing Method for Detecting Unknown Intent of Dialogue System via Pre-trained Deep Neural Network Classifier
论文作者
论文摘要
随着对话系统的成熟度和普及，检测用户在对话系统中的未知意图已成为一项重要任务。这也是最具挑战性的任务之一，因为我们几乎无法获得示例，先验知识或未知意图的确切数量。在本文中，我们提出了Softermax和Deep Nepperty Tection（SMDN），这是一种简单而有效的后处理方法，用于检测基于预训练的深度神经网络分类器的对话系统中未知的意图。我们的方法可以灵活地应用于在不改变模型体系结构的情况下在深神网络中训练的任何分类器的顶部。我们校准了软马克斯输出的置信度，以计算校准的置信得分（即SOFTERMAX），并使用它来计算未知意图检测的决策边界。此外，我们将深层神经网络学到的特征表示形式馈送到传统的新颖性检测算法中，以从不同的角度检测未知的意图。最后，我们结合了上述方法以执行关节预测。我们的方法将与已知意图不同的示例分类为未知，并且不需要任何示例或先验知识。我们已经对三个基准对话数据集进行了广泛的实验。结果表明，与最先进的基线相比，我们的方法可以产生重大改进
With the maturity and popularity of dialogue systems, detecting user's unknown intent in dialogue systems has become an important task. It is also one of the most challenging tasks since we can hardly get examples, prior knowledge or the exact numbers of unknown intents. In this paper, we propose SofterMax and deep novelty detection (SMDN), a simple yet effective post-processing method for detecting unknown intent in dialogue systems based on pre-trained deep neural network classifiers. Our method can be flexibly applied on top of any classifiers trained in deep neural networks without changing the model architecture. We calibrate the confidence of the softmax outputs to compute the calibrated confidence score (i.e., SofterMax) and use it to calculate the decision boundary for unknown intent detection. Furthermore, we feed the feature representations learned by the deep neural networks into traditional novelty detection algorithm to detect unknown intents from different perspectives. Finally, we combine the methods above to perform the joint prediction. Our method classifies examples that differ from known intents as unknown and does not require any examples or prior knowledge of it. We have conducted extensive experiments on three benchmark dialogue datasets. The results show that our method can yield significant improvements compared with the state-of-the-art baselines
