用压缩思想统一高效推理,让大模型少算却更准。
Reasoning as Compression: Unifying Budget Forcing via the Conditional Information Bottleneck
- 将推理视为信息压缩,用条件信息瓶颈约束中间思考过程
- 在不丢逻辑的前提下减少冗余计算,压缩率提升时精度下降小
- 适用于各类模型和任务,适合追求推理效率的开发者
思维链提示(CoT)虽能提升大模型在复杂任务上的准确率,但常伴随令牌使用量和推理成本上升。现有预算强制方法通过启发式长度惩罚微调来降低成本,但会同时抑制必要推理与冗余内容。本文将高效推理重新建模为信息瓶颈(IB)框架下的有损压缩问题,发现直接应用朴素IB于Transformer时,注意力机制破坏了提示、推理链与响应间的马尔可夫性。为此,提出基于条件信息瓶颈(CIB)的推理生成机制,使推理链Z仅包含从提示X无法直接获得但对响应Y至关重要的信息。该方法导出一个强化学习目标:在压缩完成文本的同时最大化任务奖励,并以推理链的先验分布为约束,涵盖常见启发式(如长度惩罚)作为特例。区别于传统按令牌数计费的方法,引入语义先验,以语言模型下的词元困惑度衡量成本,仅需查询词元级对数概率,训练开销几乎不变。实验表明,所提CIB目标能有效消除推理冗余,保持流畅性和逻辑性,在中等压缩率下提升准确率,实现高比例压缩时精度损失极小。该方法跨模型家族与任务领域表现一致,验证其作为通用思维链压缩框架的有效性。
原文摘要 · Abstract (English)
\ac{CoT} prompting improves LLM accuracy on complex tasks but often increases token usage and inference cost. Existing ``Budget Forcing'' methods reduce cost via fine-tuning with heuristic length penalties, suppressing both essential reasoning and redundant filler. We recast efficient reasoning as a lossy compression problem under the \ac{IB} principle, and identify a key theoretical gap when applying naive \ac{IB} to transformers: attention violates the Markov property between prompt, reasoning trace, and response. To resolve this issue, we model \ac{CoT} generation under the \ac{CIB} principle, where the reasoning trace $Z$ acts as a computational bridge that contains only the information about the response $Y$ that is not directly accessible from the prompt $X$. This yields a general Reinforcement Learning objective: maximize task reward while compressing completions under a prior over reasoning traces, subsuming common heuristics (e.g., length penalties) as special cases (e.g., uniform priors). In contrast to naive token-counting approaches, we introduce a semantic prior that measures token cost by surprisal under a language model. Crucially, the prior is queried only for token-level log-probabilities, adding negligible overhead to the training loop. Empirically, our \ac{CIB} objective prunes reasoning redundancy while preserving fluency and logic, improving accuracy at moderate compression and enabling aggressive compression with minimal accuracy drop. These gains generalize across model families and task domains, confirming \ac{CIB} as a domain-agnostic CoT compression framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。