提出超高压缩比思维链方法,在减少73% token的同时提升推理准确率。
Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression
- 用语义保持压缩器生成高质量监督数据,支持多压缩比训练。
- 在MATH-500上实现73%以上令牌压缩,准确率反而提升0.6%。
- 适合需要高效推理的部署场景,尤其关注数学推理任务。
思维链(CoT)推理显著提升了大语言模型的推理能力,但带来了巨大的推理计算开销。现有压缩方法在高压缩比下常导致逻辑保真度严重下降,造成性能显著退化。为实现高保真、快速推理,我们提出一种新型极端比率思维链压缩框架——Extra-CoT,大幅降低令牌预算同时保持答案准确性。首先,我们在带有细粒度标注的数学思维链数据上训练专用的语义保持压缩器,生成可靠监督信号;随后,利用混合比率监督微调(SFT)对大模型进行微调,使其学习不同压缩预算下的响应模式,并为强化学习提供稳定初始化;进一步提出约束与分层比例策略优化(CHRPO),通过分层奖励显式激励低预算下的解题能力。在三个数学推理基准上的实验表明,Extra-CoT表现优异。例如,在使用Qwen3-1.7B模型的MATH-500上,该方法实现超过73%的令牌压缩,准确率反而提升0.6%,显著优于当前最优方法。代码已开源:https://github.com/Mwie1024/Extra-CoT。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) reasoning successfully enhances the reasoning capabilities of Large Language Models (LLMs), yet it incurs substantial computational overhead for inference. Existing CoT compression methods often suffer from a critical loss of logical fidelity at high compression ratios, resulting in significant performance degradation. To achieve high-fidelity, fast reasoning, we propose a novel EXTreme-RAtio Chain-of-Thought Compression framework, termed Extra-CoT, which aggressively reduces the token budget while preserving answer accuracy. To generate reliable, high-fidelity supervision, we first train a dedicated semantically-preserved compressor on mathematical CoT data with fine-grained annotations. An LLM is then fine-tuned on these compressed pairs via a mixed-ratio supervised fine-tuning (SFT), teaching it to follow a spectrum of compression budgets and providing a stable initialization for reinforcement learning (RL). We further propose Constrained and Hierarchical Ratio Policy Optimization (CHRPO) to explicitly incentivize question-solving ability under lower budgets by a hierarchical reward. Experiments on three mathematical reasoning benchmarks show the superiority of Extra-CoT. For example, on MATH-500 using Qwen3-1.7B, Extra-CoT achieves over 73\% token reduction with an accuracy improvement of 0.6\%, significantly outperforming state-of-the-art (SOTA) methods. Our source codes have been released at https://github.com/Mwie1024/Extra-CoT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。