arXiv:2606.13316cs.AI2026-06

让大模型自动总结推理过程,更准且更省上下文。

ReSum: Synergizing LLM Reasoning and Summarization with Reinforcement Learning

论文配图:ReSum: Synergizing LLM Reasoning and Summarization with Reinforcement Learning
图 1 · 摘自论文原文
  • 用自总结机制动态压缩推理路径,避免冗长耗资源。
  • 平均提升4%准确率,推理长度减少18.6%。
  • 适合需要高效长程推理的场景,如复杂问答与规划。

强化学习结合可验证奖励(RLVR)是提升大语言模型长时推理能力的核心技术。然而,现有方法常引发不必要的长推理轨迹,降低连贯性并耗尽上下文容量。现有长上下文组织依赖外部机制,未能让模型自主管理推理过程。为此,我们提出 ReSum,一种新型 RLVR 框架,使大模型通过自总结来压缩和组织推理轨迹。初步研究表明,自总结能降低词级别熵值,稳定生成;引入“总结”短语可显著缓解错误前缀传播。受此启发,ReSum 采用感知总结的自适应推理机制:当模型自发触发总结时,遮蔽该短语形成对比分支;在非总结位置,则随机注入短语构造匹配分支。我们进一步设计了感知总结的优势函数,实现对对比推理轨迹的细粒度评估。大量实验表明,ReSum 在平均提升 4% 性能的同时,将推理长度减少 18.6%。

原文摘要 · Abstract (English)

Reinforcement Learning with Verifiable Rewards (RLVR) is a central technique for improving long-horizon reasoning in Large Language Models (LLMs). However, existing RLVR methods often encourage unnecessarily long reasoning rollouts, which can degrade reasoning coherence and exhaust the available context budget. Existing approaches to long-context organization often depend on external mechanisms to organize rollouts, rather than enabling the model to manage its own reasoning trajectory. To address this limitation, we propose ReSum, a novel RLVR framework that enables LLMs to compress and organize their reasoning trajectories through self-summarization. Our pilot studies show that self-summarization stabilizes generation by lowering token-level entropy, and that introducing a ``summarization'' phrase can substantially mitigate errors propagated from an incorrect rollout prefix. Motivated by these findings, ReSum adopts a summarization-aware adaptive rollout mechanism that contrastively evaluates whether self-summarization benefits the ongoing reasoning process. Specifically, when the model spontaneously triggers self-summarization, ReSum masks the summarization phrase to create a contrastive branch; for non-summarization positions, it instead randomly injects the phrase to create a matched branch. We further design a summarization-aware advantage to enable finer-grained comparison between contrastive rollout trajectories. Extensive experiments show that ReSum improves performance at an average of 4\% while reducing rollout length by 18.6\%.

大模型推理强化学习自总结长序列

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。