提出自适应压缩框架ZipRL，提升大模型多轮任务的上下文处理效率。
ZipRL: Adaptive Multi-Turn Context Compression with Hindsight Response Replay
- 采用粗粒度到细粒度的分层压缩策略，动态保留关键信息。
- 在5个代理任务上，相比顶尖方法性能提升27.9%~34.7%，且极低耗能。
- 适合需要长对话上下文压缩的智能体系统开发与部署场景。
自适应上下文压缩对将大语言模型扩展至复杂多轮代理任务至关重要。然而，基于规则的压缩方法可能丢弃任务关键细节，而强化学习方法在长周期任务固有的稀疏奖励下，难以平衡信息保留与标记效率。为此，我们提出针对可验证奖励强化学习（RLVR）的新型自适应压缩框架ZipRL。ZipRL包含多粒度压缩机制，实现主动、非均匀的信息削减，并引入事后响应重放（HRR）技术以增强训练信号密度。理论上，我们证明了其相较均匀方法具有更优的任务相关效用。具体而言，采用从粗到细的提示进行宏观压缩，并通过广义优势重塑将HRR融入GRPO算法。多个不同版本与参数规模的模型验证了该方法的有效性。在五个代理任务上的基准测试显示，相较于现有最优方法，ZipRL在Qwen3-4B和Qwen3-8B模型上分别提升27.9%和34.7%，同时在极端256轮外推压力测试中保持卓越的令牌效率与鲁棒性。
原文摘要 · Abstract (English)
Adaptive context compression is vital for scaling Large Language Models (LLMs) to complex, multi-turn agent tasks. However, rule-based compression methods may discard task-critical nuances, while Reinforcement Learning (RL) approaches usually struggle to balance information retention and token efficiency under the sparse rewards inherent to long-horizon workflows. To bridge this gap, we propose ZipRL, a novel adaptive compression framework tailored for Reinforcement Learning from Verifiable Rewards (RLVR). ZipRL features a multi-granularity compression mechanism for active, non-uniform information reduction, coupled with Hindsight Response Replay (HRR), a technique designed to densify training signals during RLVR optimization. Theoretically, we prove ZipRL's superior task-relevant utility over uniform methods. Concretely, ZipRL utilizes coarse-to-fine prompts for macro-compression and incorporates HRR into GRPO via generalized advantage reshaping. Multiple models of varying versions and parameter scales validate the effectiveness of our approach. Benchmarks on five agent tasks show ZipRL outperforms state-of-the-art approaches by 27.9% and 34.7% across Qwen3-4B and Qwen3-8B models, while maintaining exceptional token efficiency and robustness under extreme 256-turn extrapolation stress tests.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。