通过复用历史信息提升长对话中大模型的推理效率。
Context Recycling for Long-Horizon LLM Inference
- 用结构化查询与外部记忆实现关键信息复用。
- 15轮对话中减少令牌消耗，保持回答准确率。
- 适合需要长期交互的医疗等复杂场景使用。
大语言模型在短上下文推理中表现优异，但在长对话中因上下文窗口限制和令牌使用效率低而性能下降。我们提出ContextForge系统，通过结构化查询生成、外部记忆检索和可控合成，在多轮对话中保留任务相关知识，实现无需完整重放上下文的高效计算复用，显著降低令牌开销的同时维持答案质量。在包含15轮对话的医疗结构化查询基准测试中，相比基线模型，ContextForge提升了推理一致性并减少令牌消耗，且保持相近的响应准确率。结果表明，上下文复用是一种无需扩大上下文窗口或重新训练模型即可拓展长时序任务能力的实用方案。代码与评估数据已开源。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit strong capabilities in short-context reasoning but degrade in performance over long conversational horizons due to context window limitations and inefficient token usage. We introduce ContextForge, a system for context recycling that maintains task-relevant information across turns by combining structured query generation, external memory retrieval, and controlled synthesis. The system enables efficient reuse of prior computation without relying on full context replay, reducing token overhead while preserving answer quality. We evaluate ContextForge using a 15-turn conversational benchmark that tests multi-turn reasoning, back-references, and domain shifts across structured healthcare queries. Compared to a baseline agent using identical underlying models, ContextForge demonstrates improved consistency and reduced token consumption, while maintaining comparable response accuracy. These results suggest that context recycling provides a practical approach for extending LLM capabilities in long-horizon tasks without requiring larger context windows or model retraining. Code and evaluation artifacts are available at https://github.com/Betanu701/ContextForge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。