上下文中的错误会误导大模型，导致后续推理一错再错。
Contextual Drag: How Errors in the Context Affect LLM Reasoning
- 用树编辑距离分析发现错误模式在上下文中复制
- 错误上下文使模型性能下降10%-20%，自修正反而恶化
- 即使有正确反馈也难消除，适合研究推理鲁棒性者看
大语言模型自改进流程依赖于对过往错误的反思。本文研究了一种称为上下文拖拽的现象：上下文中存在的失败尝试会使后续生成倾向于产生结构相似的错误。在8个推理任务上对11个专有及开源模型的评估显示，上下文拖拽导致性能下降10%-20%，且严重受此影响的模型在迭代自修正过程中会陷入自我恶化。通过树编辑距离的结构分析表明，后续推理路径继承了上下文中的错误模式。我们证明，外部反馈或成功自验证均无法消除该效应。尽管采用回退行为微调和上下文去噪等缓解策略可部分改善表现，但无法完全恢复基线性能，表明上下文拖拽是当前推理架构中一个持续存在的失效模式。
原文摘要 · Abstract (English)
Central to many self-improvement pipelines for large language models (LLMs) is the assumption that models can improve by reflecting on past mistakes. We study a phenomenon termed contextual drag: the presence of failed attempts in the context biases subsequent generations toward structurally similar errors. Across evaluations of 11 proprietary and open-weight models on 8 reasoning tasks, contextual drag induces 10-20% performance drops, and iterative self-refinement in models with severe contextual drag can collapse into self-deterioration. Structural analysis using tree edit distance reveals that subsequent reasoning trajectories inherit structurally similar error patterns from the context. We demonstrate that neither external feedback nor successful self-verification suffices to eliminate this effect. While mitigation strategies such as fallback-behavior fine-tuning and context denoising yield partial improvements, they fail to fully restore baseline performance, positioning contextual drag as a persistent failure mode in current reasoning architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。