用摘要压缩解决大模型长序列探索难题,无需重训练
ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization
- 通过外部工具定期压缩交互历史,实现无限制探索
- 在不训练情况下比ReAct提升4.5%,结合训练再增8.2%
- 仅用1000样本即可让30B模型媲美顶尖开源模型
基于大语言模型的网络代理在知识密集型任务中表现优异,但面临广泛探索需求与有限上下文窗口之间的根本矛盾。现有方法多依赖架构修改(如内部记忆令牌),破坏与已有代理的兼容性,并需昂贵的端到端重训练。为此,我们提出ReSum,一种轻量级、即插即用的范式,通过定期调用外部工具将交互历史压缩为紧凑摘要,实现无界探索。尽管该范式无需训练,但标准代理无法自然推理此类压缩上下文。为此,我们提出ReSum-GRPO,通过优势广播适配组相对策略优化(GRPO),在分段轨迹间传播最终奖励,实现长时程信用分配。大量实验表明,ReSum在无训练设置下比ReAct提升4.5%,而ReSum-GRPO进一步提升8.2%。值得注意的是,仅用1000个训练样本,一个增强后的30B代理即可达到与领先开源模型相当的性能,验证了ReSum的有效性。
原文摘要 · Abstract (English)
Large Language Model (LLM)-based web agents excel at knowledge-intensive tasks but face a fundamental conflict between the need for extensive exploration and the constraints of limited context windows. Current solutions typically rely on architectural modifications, e.g., internal memory tokens, which break compatibility with pre-existing agents and necessitate costly end-to-end retraining. To overcome these limitations, we introduce ReSum, a lightweight, plug-and-play paradigm that enables unbounded exploration by periodically invoking an external tool to condense interaction histories into compact summaries. Although this paradigm functions without training, standard agents are not inherently aligned to reason over such compressed contexts. To bridge this gap, we propose ReSum-GRPO, which adapts Group Relative Policy Optimization (GRPO) via advantage broadcasting to propagate final rewards across segmented trajectories, enabling credit assignments over long-horizons. Extensive experiments show that ReSum achieves a 4.5% improvement over ReAct in training-free settings, with ReSum-GRPO yielding a further 8.2% gain. Notably, with only 1K training samples, a ReSum-enhanced 30B agent achieves competitive performance with leading open-source models, showing ReSum's effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。