让大模型在逐步对话中也能准确推理，避免因信息碎片化而出错。
Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models
- 用教师-学生框架对齐完整提示与分步对话的推理路径
- 数学任务上分步输入性能提升32%，跨领域零样本任务也显著改善
- 适合需要多轮交互、高可信度推理的应用场景
大语言模型在一次性给出完整指令时表现良好，但在信息逐步揭示的多轮对话中常失败。当完整提示（FULL）与分段对话（RAW-SHARDED）包含相同用户证据时，模型应得出一致答案。我们指出关键原因是自锚定漂移：部分信息下的回应引入无依据假设，进而扭曲最终结果。为此提出规范上下文在线策略蒸馏（CCOPD）：固定教师模型基于完整提示推理，可训练学生模型逐轮接收相同信息；通过对其自身轨迹与教师行为对齐，减少偏差。仅在数学问题对话上训练，CCOPD在数学及五个零样本跨领域任务族上，分步输入性能相比基线模型平均提升32%，同时保持完整提示性能。分析显示，该方法增强了对用户证据的依赖性，降低了早期回复污染的影响。
原文摘要 · Abstract (English)
Large language models (LLMs) often solve a task when all instructions are given in a single prompt, but fail when the same information is revealed gradually across turns. When a clean FULL prompt and a RAW-SHARDED conversation contain the same complete user evidence, the model should still arrive at the same answer. We argue that a key reason for this gap is self-anchored drift: responses produced under partial information introduce unsupported assumptions, and those assumptions later distort the final answer. To reduce this effect, we propose Canonical-Context On-Policy Distillation (CCOPD). During training, the same base model is used in two roles: a frozen teacher conditioned on the clean FULL prompt and a trainable student that receives the same evidence incrementally through a multi-turn conversation; CCOPD aligns the student's behavior on its own trajectories with the teacher's canonical full-context behavior. Trained only on math problem conversations, CCOPD yields a 32\% average relative improvement in RAW-SHARDED performance over the original base model across math and five zero-shot out-of-domain task families, while largely preserving full-context performance. Further analyses suggest that CCOPD strengthens grounding in user evidence and reduces sensitivity to contamination from earlier assistant turns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。