用强化学习构建记忆代理，实现超长文本处理且性能几乎不降
MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent
- 设计多回合生成训练机制，通过重写策略更新记忆
- 从32K上下文训练，可拓展至350万词问答任务，性能损失<5%
- 在51.2万词测试集上达到95%以上准确率，适合超长文档任务
尽管已有长度外推、高效注意力和记忆模块的改进，但在线性复杂度下处理无限长文档且外推时性能不下降仍是长期文本处理的终极挑战。本文直接以端到端方式优化长文本任务，提出新型代理流程MemAgent，将文本分段读取并采用覆盖策略更新记忆。我们扩展了DAPO算法，支持通过独立上下文多轮对话生成进行训练。MemAgent展现出卓越的长上下文能力：从32K文本训练的8K上下文，可外推至350万词的问答任务，性能损失低于5%；在51.2万词的RULER测试中准确率超过95%。
原文摘要 · Abstract (English)
Despite improvements by length extrapolation, efficient attention and memory modules, handling infinitely long documents with linear complexity without performance degradation during extrapolation remains the ultimate challenge in long-text processing. We directly optimize for long-text tasks in an end-to-end fashion and introduce a novel agent workflow, MemAgent, which reads text in segments and updates the memory using an overwrite strategy. We extend the DAPO algorithm to facilitate training via independent-context multi-conversation generation. MemAgent has demonstrated superb long-context capabilities, being able to extrapolate from an 8K context trained on 32K text to a 3.5M QA task with performance loss < 5% and achieves 95%+ in 512K RULER test.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。