让大模型的思考过程自动压缩上下文,提升长文本问答效率。
Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor

- 用模型思考过程替代专用压缩模块,生成精简上下文。
- 在4倍和8倍压缩比下,F1得分领先17.4%和23.4%。
- 无需额外训练,适合部署在现有推理系统中。
上下文压缩旨在以最小信息损失缩短长输入,加速大语言模型推理。现有方法多依赖复杂压缩模块或特定训练,未充分挖掘大模型自身的压缩潜力。本文揭示,思考模型本身能通过组织任务相关资讯自然压缩长上下文,提出『思考即压缩』(TaC)新范式,将思考过程视为压缩后的上下文。无需专用压缩器,直接提示模型生成思考轨迹作为简化输入,性能已超越多数主流压缩方法。进一步针对原始思考输出存在预算失控与捷径行为问题,提出受限型思考即压缩(TaC-C),采用简单奖励驱动优化框架,引导模型生成紧凑可控的压缩内容。在四个长上下文问答基准测试中,TaC-C持续优于现有基线:在4倍与8倍压缩比下,平均F1分别高出17.4%与23.4%,平均精确匹配率(EM)分别高出15.7%与21.7%。
原文摘要 · Abstract (English)
Context compression aims to shorten long context inputs with minimal information loss for LLM inference acceleration. While existing methods have shown promise, they typically rely on complex compression modules or compression-specific training, leaving the intrinsic capabilities of LLMs underexplored. In contrast, this work reveals that a thinking model itself can naturally compress long contexts by organizing task-relevant information. We thus derive Thinking as Compression (TaC), a new compression paradigm that treats thinking itself as compressed context. Without relying on specific dedicated compressor, TaC directly prompts the thinking model to generate thinking traces as the shortened context, already outperforming most representative compression methods. Further, given that raw thinking output may struggle with budget control and shortcut behaviors, we introduce Thinking as Compression Constrained (TaC-C), leveraging a simple reward-driven optimization framework to elicit intrinsic thinking as compact and controllable compressed context. Experiments across four long-context QA benchmarks demonstrate that TaC-C consistently outperforms existing baselines. At 4x and 8x compression ratios, it surpasses the strongest competitor by 17.4% and 23.4% in average F1, and by 15.7% and 21.7% in average Exact Match Score (EM), respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。