论文标题
部分可观测时空混沌系统的无模型预测
Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcity
论文作者
论文摘要
我们研究了通过低资源语言镜头来确定神经分类器的预测信心（或相反的不确定性）的问题。通过对三种不同语言的子采样数据集的培训模型，我们评估了各种方法的估计质量及其对可用数据量的依赖。我们发现，尽管基于预先训练的模型和合奏的方法总体上取得了最佳结果，但不确定性估计的质量可能会遭受更多数据的影响。我们还对序列的不确定性进行了定性分析，发现模型的总不确定性似乎受其数据不确定性的影响很大，而不是模型不确定性。所有模型实现都在软件包中开源。
We investigate the problem of determining the predictive confidence (or, conversely, uncertainty) of a neural classifier through the lens of low-resource languages. By training models on sub-sampled datasets in three different languages, we assess the quality of estimates from a wide array of approaches and their dependence on the amount of available data. We find that while approaches based on pre-trained models and ensembles achieve the best results overall, the quality of uncertainty estimates can surprisingly suffer with more data. We also perform a qualitative analysis of uncertainties on sequences, discovering that a model's total uncertainty seems to be influenced to a large degree by its data uncertainty, not model uncertainty. All model implementations are open-sourced in a software package.
