论文标题
关于概率分类器集的校准
On the Calibration of Probabilistic Classifier Sets
论文作者
论文摘要
产生一组概率分类器（例如集合学习方法）的多类分类方法能够模拟质地和认知不确定性。然后，通常通过贝叶斯误差来量化核心的不确定性，并通过集合的大小来量化认知不确定性。在本文中，我们扩展了校准的概念，该概念通常用于评估单个概率分类器的息肉不确定性表示的有效性，以评估通过概率分类器集获得的认知不确定性表示的有效性。从广义上讲，如果可以找到这些分类器的校准凸组组合，我们称之为校准的一组概率分类器。为了评估这种校准概念，我们提出了一种新型的非参数校准测试，该测试将现有的单个概率分类器的现有测试推广到一组概率分类器的情况下。利用该测试，我们从经验上表明，深度神经网络的集合通常无法很好地校准。
Multi-class classification methods that produce sets of probabilistic classifiers, such as ensemble learning methods, are able to model aleatoric and epistemic uncertainty. Aleatoric uncertainty is then typically quantified via the Bayes error, and epistemic uncertainty via the size of the set. In this paper, we extend the notion of calibration, which is commonly used to evaluate the validity of the aleatoric uncertainty representation of a single probabilistic classifier, to assess the validity of an epistemic uncertainty representation obtained by sets of probabilistic classifiers. Broadly speaking, we call a set of probabilistic classifiers calibrated if one can find a calibrated convex combination of these classifiers. To evaluate this notion of calibration, we propose a novel nonparametric calibration test that generalizes an existing test for single probabilistic classifiers to the case of sets of probabilistic classifiers. Making use of this test, we empirically show that ensembles of deep neural networks are often not well calibrated.
