研究大模型对话中历史回复如何影响后续表现，发现删掉模型自己说的话能提升性能。
What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction
- 通过重放用户输入并替换模型历史回复，测试其对结果的影响
- 替换历史回复使平均性能提升0.027，短文本效果与长文本相当
- 约64%的对话可通过修改某一轮回复改善，适合需要稳定交互的场景
多轮对话中，大模型的先前回复会成为后续行为的上下文，形成反馈循环。现有研究显示多轮性能下降明显，且助手生成的历史会影响后续表现，但其影响在不同模型、任务、轮次和内部状态中的具体机制尚不明确。本文在六类任务和五种模型上展开研究：从全量单轮输入（FULL）到逐步揭示的多轮交互（SHARDED），性能下降具有任务和模型依赖性，初始表现好并不意味着交互更鲁棒。随后，我们回溯已完成的SHARDED对话，仅编辑助手生成的历史内容而重放用户消息。将历史替换为中性内容（称为中性化）后，在2,973条轨迹上，下游最小-最大归一化性能平均提升+0.027。在预设长度控制的子集上，短版本与长度匹配版本中性化效果几乎一致（+0.069对比+0.068），说明单纯缩短上下文无法解释该效应。进一步的“轮次手术”实验表明，在237个性能下降的轨迹中，63.7%存在至少一次有益干预，多数位置仍无变化；对于二分类任务，48.4%可实现失败到成功的逆转。开放权重案例研究将行为显著变化与可测量的下游状态差异关联，但发现这些差异具有任务依赖性而非普遍规律。总体而言，助手生成的历史具有主动但选择性的影响力，提示应采取选择性而非统一的方式管理历史。
原文摘要 · Abstract (English)
Multi-turn interaction creates a feedback process in which an LLM's previous responses become context for later behavior. Prior work shows substantial multi-turn degradation and that assistant-generated history can affect later behavior. However, it remains unclear how these effects manifest across models, tasks, turns, and inside a model. We study these gaps across six task families and five models. Degradation from fully specified single-turn input (FULL) to progressively revealed multi-turn interaction (SHARDED) is clearly task- and model-dependent, and stronger one-shot performance does not imply greater interaction robustness. We then retrospectively analyze completed SHARDED conversations by replaying the user messages already observed in each trajectory while editing only assistant-generated history. Replacing prior assistant responses with neutral content (termed neutralization) changes downstream min-max normalized performance by +.027 across 2,973 trajectories. On a prespecified length-controlled subset, short and length-matched neutralization yield nearly identical effects (+.069 versus +.068), showing that simple context shortening is insufficient to explain the effect of history editing. Turn Surgery further intervenes on one assistant turn at a time. Among 237 selected degraded trajectories, 63.7% contain at least one beneficial intervention, while most tested positions remain unchanged; for binary tasks, 48.4% admit a fail-to-success reversal. An open-weight case study links behaviorally consequential history changes to measurable downstream state differences, but finds task-dependent rather than universal internal signatures. Overall, assistant-generated history has active but selective effects on multi-turn performance, motivating selective rather than uniform history management.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。