研究大模型在含冲突演示时的推理失效机制，发现早期注意力头易受干扰。
Understanding the Dynamics of Demonstration Conflict in In-Context Learning
- 通过规则推断任务分析模型对矛盾示例的处理方式。
- 单个错误示例使模型性能大幅下降，中间层同时编码正确与错误规则。
- 定位到关键注意力头，针对性删除可提升性能超10%。
上下文学习使大语言模型能通过少量示例完成新任务，但示例本身可能包含噪声或矛盾，导致能力脆弱。为理解模型如何处理此类冲突，我们研究需推理隐藏模式的任务，即规则推断。发现仅一个规则被破坏的示例就导致模型性能显著下降，引发系统性误导行为。通过线性探针与逻辑透镜分析，揭示模型在中间层同时编码正确与错误规则，而仅在后期层建立预测信心，呈现两阶段计算结构。进一步识别出对应各阶段的关键注意力头：早期至中期的‘易损头’表现出位置敏感性，对污染高度敏感；后期的‘易感头’在接触错误示例时显著削弱对正确预测的支持。定向消融验证结果，仅掩蔽少量识别出的头即可使性能提升超过10%。
原文摘要 · Abstract (English)
In-context learning enables large language models to perform novel tasks through few-shot demonstrations. However, demonstrations per se can naturally contain noise and conflicting examples, making this capability vulnerable. To understand how models process such conflicts, we study demonstration-dependent tasks requiring models to infer underlying patterns, a process we characterize as rule inference. We find that models suffer substantial performance degradation from a single demonstration with corrupted rule. This systematic misleading behavior motivates our investigation of how models process conflicting evidence internally. Using linear probes and logit lens analysis, we discover that under corruption models encode both correct and incorrect rules in intermediate layers but develop prediction confidence only in late layers, revealing a two-phase computational structure. We then identify attention heads for each phase underlying the reasoning failures: Vulnerability Heads in early-to-middle layers exhibit positional attention bias with high sensitivity to corruption, while Susceptible Heads in late layers significantly reduce support for correct predictions when exposed to the corrupted evidence. Targeted ablation validates our findings, with masking a small number of identified heads improving performance by over 10%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。