论文标题
地面真实标签很重要：更深入地研究输入标签演示
Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations
论文作者
论文摘要
尽管最近在秘密学习中兴趣爆炸，但示范质量的基本机制和确切影响仍然难以捉摸。从直觉上，地面真实标签在内在学习（ICL）中的影响应与受到监督学习一样多，但是最近的工作报告说，输入标签的对应关系的重要性明显低于以前想象的。对这种违反直觉的观察，我们重新检查了地面标签在封闭式学习中的重要性。随着两个新型指标的引入，即标记校正性敏感性和地面标签效应比（GLER），我们能够对地面真实标签示范的影响进行可量化的分析。通过广泛的分析，我们发现正确的输入标签映射映射可能会对下游的内在学习性能产生不同的影响，具体取决于实验配置。通过其他研究，我们将关键组成部分（例如及时模板的详细性和语言模型大小）确定为实现更多降噪的ICL的控制因素。
Despite recent explosion of interests in in-context learning, the underlying mechanism and the precise impact of the quality of demonstrations remain elusive. Intuitively, ground-truth labels should have as much impact in in-context learning (ICL) as supervised learning, but recent work reported that the input-label correspondence is significantly less important than previously thought. Intrigued by this counter-intuitive observation, we re-examine the importance of ground-truth labels in in-context learning. With the introduction of two novel metrics, namely Label-Correctness Sensitivity and Ground-truth Label Effect Ratio (GLER), we were able to conduct quantifiable analysis on the impact of ground-truth label demonstrations. Through extensive analyses, we find that the correct input-label mappings can have varying impacts on the downstream in-context learning performances, depending on the experimental configuration. Through additional studies, we identify key components, such as the verbosity of prompt templates and the language model size, as the controlling factor to achieve more noise-resilient ICL.
