解决对话中模型走偏问题，让大模型更持久保持准确
MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillation
- 用自净化历史重写对话中间回复，减少信息污染
- 在7B模型上将分段准确率从52.8%提升至66.1%
- 无需额外标注或推理结构，适合实际对话系统部署
大语言模型在单次指令下表现良好，但在多轮对话中会逐渐偏离，称为‘对话迷失’（LiC）现象。我们发现部分原因在于自我污染：早期助手回复被保留进后续上下文，导致偏差累积。为此提出MAIGO方法，通过模型自身策略生成的干净历史参考，进行在线自蒸馏。在中间轮次中清除历史助手回复，仅保留用户可见的分段前缀；在回答轮次则使用完整视角参考进行蒸馏，并引入可靠性权重过滤与干净参考不一致的样本。该方法无需验证奖励、状态标签或推理时辅助结构。在带确定性验证器的LiC配对视图协议下，Qwen2.5-7B-Instruct模型的分段准确率从52.8%提升至66.1%，分段/完整准确率比从66.5%升至84.1%，完整准确率仅下降2.3个百分点。结果表明，自我污染是可训练的对话迷失组成部分。
原文摘要 · Abstract (English)
Large language models often solve tasks from a fully specified prompt but degrade when the same requirements unfold over multiple turns, known as the lost-in-conversation (LiC) gap. We trace part of this degradation to self-contamination: intermediate assistant replies enter later context and carry early deviations forward. Motivated by this mechanism, we propose MAIGO, an on-policy self-distillation method that reduces this contamination using history-cleaned references from the model's own policy. For middle turns, MAIGO removes prior assistant replies while preserving the user-visible sharded prefix; for answer turns, it distills from paired full-view references conditioned on the completed user-side dialogue. A reliability weight downweights middle-turn samples that disagree with the clean reference. MAIGO requires no verifier rewards, state labels, or inference-time scaffolding. Under the LiC paired-view protocol with deterministic verifiers, MAIGO improves Qwen2.5-7B-Instruct SHARDED accuracy from 52.8 to 66.1 and the SHARDED/FULL ratio from 66.5% to 84.1%, while keeping FULL accuracy within 2.3 points. These results show that self-contamination is a trainable component of the LiC gap.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。