通过引导熵变化压缩大模型推理链,兼顾效率与准确率。
Entropy-Guided Reasoning Compression
- 设计熵引导训练框架,动态调节推理长度与探索度。
- 在六项数学任务上将推理长度压缩至原长的20%,精度不降反升。
- 适合追求高效推理部署的研究者与工程师使用。
大型推理模型在复杂推理任务中表现优异,但其思维链输出过长,导致计算开销大、部署困难。现有压缩方法虽部分成功,却忽略了训练过程中的熵冲突现象:压缩训练中熵下降使推理变短但探索不足,而以准确率为目标的训练则提升熵,延长推理链。这种矛盾使模型陷入局部困境。分析发现,高熵词多为逻辑连接词,它们在性能目标下获得更大梯度被鼓励,却在压缩目标下被惩罚,形成直接的熵冲突源。为此,我们提出熵引导训练框架:当熵下降时,引导模型向简洁推理迈进;当熵上升时,在紧凑模式下强化探索以提升鲁棒性。在六个数学基准上的实验表明,该方法可将推理长度压缩至原始长度的20%,同时保持甚至超越基线准确率。代码与模型将公开发布。
原文摘要 · Abstract (English)
Large reasoning models have demonstrated remarkable performance on complex reasoning tasks, yet the excessive length of their chain-of-thought outputs remains a major practical bottleneck due to high computation cost and poor deployability. Existing compression methods have achieved partial success but overlook a crucial phenomenon in the training process -- the entropy conflict. During compression training, entropy decreases, leading to shorter reasoning but limited exploration, while accuracy-oriented objectives increase entropy, lengthening reasoning chains. This can cause the model to get stuck in a local dilemma. Our analysis further reveals the origin of the entropy conflict: many high-entropy tokens are logical connectors that receive larger gradients and are encouraged under the performance objective, while the compression objective simultaneously penalizes these potentially redundant connectors. This opposing pressure creates a direct source of entropy conflict. To address these issues, we adopt an entropy-guided training framework. As entropy descends, the model is guided toward efficient reasoning by encouraging concise thought steps; as entropy rises, exploration is reinforced under the compact reasoning mode to improve robustness. Experiments on six mathematical benchmarks show that our method compresses reasoning length to 20% of the original while maintaining or even surpassing baseline accuracy. Code and models will be released publicly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。