论文标题
动作得分的无监督难度估计
Unsupervised Difficulty Estimation with Action Scores
论文作者
论文摘要
随着现实世界中的当前模型现在正在应用，评估机器学习模型中的难度和偏见已变得非常重要。在本文中，我们提出了一种基于训练过程中每个样本的损失的积累来计算难度得分的简单方法。我们将其称为动作分数。我们提出的方法不需要对模型进行任何任何外部监督的任何修改，因为它可以作为回调实现，从而从培训过程中收集信息。我们在两个不同的设置中测试和分析我们的方法：图像分类和对象检测，我们表明，在这两个设置中，动作分数都可以提供有关模型和数据集偏差的见解。
Evaluating difficulty and biases in machine learning models has become of extreme importance as current models are now being applied in real-world situations. In this paper we present a simple method for calculating a difficulty score based on the accumulation of losses for each sample during training. We call this the action score. Our proposed method does not require any modification of the model neither any external supervision, as it can be implemented as callback that gathers information from the training process. We test and analyze our approach in two different settings: image classification, and object detection, and we show that in both settings the action score can provide insights about model and dataset biases.
