论文标题
注意您的偏见：对上下文语言模型的偏见检测方法的批判性评论
Mind Your Bias: A Critical Review of Bias Detection Methods for Contextual Language Models
论文作者
论文摘要
对偏见的意识和缓解对于公平和透明使用上下文语言模型至关重要，但是它们至关重要地取决于对偏见作为先驱者的准确检测。因此，已经提出了许多偏差检测方法，它们的方法各不相同，所考虑的偏见类型以及用于评估的数据。但是，虽然大多数检测方法是从静态单词嵌入的单词嵌入关联测试中得出的，但报告的结果是异质的，不一致的，并且最终尚无定论。为了解决这个问题，我们对上下文语言模型进行了严格的分析和比较偏见检测方法。我们的结果表明，较小的设计和实施决策（或错误）对派生的偏差分数产生了重大且通常很大的影响。总体而言，我们发现该领域的状态既比实施中的系统和传播错误，都比以前所承认的要差，但比预期的要好，因为在考虑实施错误后，文献中文献的不同结果均匀化。根据我们的发现，我们以讨论通往更健壮和一致的偏见检测方法的途径进行了结论。
The awareness and mitigation of biases are of fundamental importance for the fair and transparent use of contextual language models, yet they crucially depend on the accurate detection of biases as a precursor. Consequently, numerous bias detection methods have been proposed, which vary in their approach, the considered type of bias, and the data used for evaluation. However, while most detection methods are derived from the word embedding association test for static word embeddings, the reported results are heterogeneous, inconsistent, and ultimately inconclusive. To address this issue, we conduct a rigorous analysis and comparison of bias detection methods for contextual language models. Our results show that minor design and implementation decisions (or errors) have a substantial and often significant impact on the derived bias scores. Overall, we find the state of the field to be both worse than previously acknowledged due to systematic and propagated errors in implementations, yet better than anticipated since divergent results in the literature homogenize after accounting for implementation errors. Based on our findings, we conclude with a discussion of paths towards more robust and consistent bias detection methods.
