论文标题
贝叶斯近似的分布校准用于域外检测
Distribution Calibration for Out-of-Domain Detection with Bayesian Approximation
论文作者
论文摘要
室外（OOD）检测是以任务为导向的对话框系统中的关键组件，旨在确定查询是否不在预定义的支持的意图集之外。事实证明，先前基于软疗法的检测算法对OOD样品被过度自信。在本文中，我们分析了过度自信的OOD来自由于训练和测试分布之间的不匹配而导致的分布不确定性，这使得该模型无法自信地做出预测，因此可能导致异常软磁得分。我们提出了一个贝叶斯OOD检测框架，以使用Monte-Carlo辍学来校准分布不确定性。我们的方法是灵活的，并且与MSP相比仅增加了0.41 \％的推理时间，因此在现有的基于软敏的基线和增益中易于插入33.33 \％OOD F1的改进。进一步的分析表明，贝叶斯学习对OOD检测的有效性。
Out-of-Domain (OOD) detection is a key component in a task-oriented dialog system, which aims to identify whether a query falls outside the predefined supported intent set. Previous softmax-based detection algorithms are proved to be overconfident for OOD samples. In this paper, we analyze overconfident OOD comes from distribution uncertainty due to the mismatch between the training and test distributions, which makes the model can't confidently make predictions thus probably causing abnormal softmax scores. We propose a Bayesian OOD detection framework to calibrate distribution uncertainty using Monte-Carlo Dropout. Our method is flexible and easily pluggable into existing softmax-based baselines and gains 33.33\% OOD F1 improvements with increasing only 0.41\% inference time compared to MSP. Further analyses show the effectiveness of Bayesian learning for OOD detection.
