让大模型学会分段记忆，提升多轮对话中的推理准确率
Multi-Turn Reasoning When Context Arrives in Pieces: Scalable Sharding and Memory-Augmented RL
- 用滚动记忆替代完整历史，减少信息丢失
- 在分片GSM8K上训练后，多轮推理准确率提升65%
- 适合需要长上下文推理的复杂任务场景
当用户在多轮对话中分段提供关键信息时，即使上下文完整可用，大模型的准确率仍会下降高达65%。我们发现，通过训练模型维持一个紧凑的滚动记忆，而非依赖不断增长的历史，可显著缓解这一‘对话中迷失’问题。为实现训练的可扩展性，我们提出一种低成本分片流水线，将单轮问答数据集转换为包含碎片化信息的多轮对话片段，无需数小时的人工标注。仅在分片GSM8K上训练的带记忆策略，在多轮任务中显著提升准确率，并零样本泛化至更难的数学题和跨领域的长上下文问答。此外，即使测试时提供完整历史，记忆训练模型的表现仍优于全历史基线，表明学习压缩信息能带来比单纯暴露完整上下文更鲁棒的增量推理能力。
原文摘要 · Abstract (English)
When a user reveals task-critical information across several conversation turns, LLM accuracy drops by up to 65% despite full context availability. We show that this Lost in Conversation degradation can be substantially mitigated by training models to maintain a compact rolling memory instead of attending to a growing history. To make such training scalable, we introduce a low-cost sharding pipeline that converts single-turn QA datasets into multi-turn fragmented-information episodes, eliminating the need for hours of manual annotation. Training only on sharded GSM8K, our memory-augmented policy significantly improves multi-turn accuracy and generalises zero-shot to harder math and out-of-domain long-context QA. Moreover, memory-trained models outperform full-history baselines even when given the full history at test time, suggesting that learning to compress induces more robust incremental reasoning than full-context exposure alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。