论文标题
可访问性图像描述的上下文事项：无参考评估指标的挑战
Context Matters for Image Descriptions for Accessibility: Challenges for Referenceless Evaluation Metrics
论文作者
论文摘要
网络上很少有图像接收Alt-Text描述，这些描述可以使它们成为盲目和低视力（BLV）用户的访问。基于图像的NLG系统已经发展到可以开始解决这个持续的社会问题的地步，但是除非我们对指标正确指导其发展的指标进行评估，否则这些系统将不会完全成功。在这里，我们反对当前的无参考指标 - 那些不依赖人类生成的基础真相描述的指标 - 理由是他们与BLV用户的需求不符。这些指标的根本缺点是它们没有考虑到上下文，而上下文信息则被BLV用户高度重视。为了证实这些主张，我们向BLV参与者介绍了一项研究，他们对各种维度进行了评分。深入的分析表明，缺乏上下文意识使当前的无引用指标不足以提高图像可访问性。作为概念验证，我们提供了无引用度量夹的上下文版本，该版本开始解决与BLV数据的断开连接。本文可访问的html版本可在https://elisakreiss.github.io/contextual-description-evaluation/paper/reflessmetrics.html上获得
Few images on the Web receive alt-text descriptions that would make them accessible to blind and low vision (BLV) users. Image-based NLG systems have progressed to the point where they can begin to address this persistent societal problem, but these systems will not be fully successful unless we evaluate them on metrics that guide their development correctly. Here, we argue against current referenceless metrics -- those that don't rely on human-generated ground-truth descriptions -- on the grounds that they do not align with the needs of BLV users. The fundamental shortcoming of these metrics is that they do not take context into account, whereas contextual information is highly valued by BLV users. To substantiate these claims, we present a study with BLV participants who rated descriptions along a variety of dimensions. An in-depth analysis reveals that the lack of context-awareness makes current referenceless metrics inadequate for advancing image accessibility. As a proof-of-concept, we provide a contextual version of the referenceless metric CLIPScore which begins to address the disconnect to the BLV data. An accessible HTML version of this paper is available at https://elisakreiss.github.io/contextual-description-evaluation/paper/reflessmetrics.html
