让大模型在多轮对话中不丢信息,自动压缩聊天记录
MT-OSC: Path for LLMs that Get Lost in Multi-Turn Conversation

- 用轻量代理自动筛选并压缩对话历史,不打扰用户
- 多轮对话可减少72%的token用量,性能更稳定
- 适合需要长对话能力的智能客服、助手等场景
大型语言模型在多轮对话中因指令与上下文分散而表现显著下降,但多轮交互是聊天界面的主流。常规做法将完整对话历史附加到提示中,迅速耗尽上下文窗口,导致延迟增加、计算成本上升且收益递减。我们提出MT-OSC,一种一次性序列压缩框架,可在后台高效自动压缩对话历史而不影响用户体验。MT-OSC采用基于少样本推理的压缩代理和轻量级决策器,选择性保留关键信息,在10轮对话中最多减少72%的令牌数量。在13个先进LLM及多种多轮基准测试中评估,MT-OSC持续缩小多轮性能差距,在数据集上保持或提升准确率,对干扰项和无关回合也具鲁棒性。结果表明,MT-OSC是多轮对话的可扩展解决方案,能在有限输入空间内实现更丰富上下文,降低延迟与运营成本,兼顾性能表现。
原文摘要 · Abstract (English)
Large language models (LLMs) suffer significant performance degradation when user instructions and context are distributed over multiple conversational turns, yet multi-turn (MT) interactions dominate chat interfaces. The routine approach of appending full chat history to prompts rapidly exhausts context windows, leading to increased latency, higher computational costs, and diminishing returns as conversations extend. We introduce MT-OSC, a One-off Sequential Condensation framework that efficiently and automatically condenses chat history in the background without disrupting the user experience. MT-OSC employs a Condenser Agent that uses a few-shot inference-based Condenser and a lightweight Decider to selectively retain essential information, reducing token counts by up to 72% in 10-turn dialogues. Evaluated across 13 state-of-the-art LLMs and diverse multi-turn benchmarks, MT-OSC consistently narrows the multi-turn performance gap - yielding improved or preserved accuracy across datasets while remaining robust to distractors and irrelevant turns. Our results establish MT-OSC as a scalable solution for multi-turn chats, enabling richer context within constrained input spaces, reducing latency and operational cost, while balancing performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。