arXiv:2511.13223cs.LGcs.AI2025-11NeurIPS被引 6

压缩大模型推理过程,省一半token还能保持准确率。

TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs

  • 自动生成数据,按题目难易动态压缩推理深度。
  • 用语言优化技术让表达更简洁清晰,逻辑不变。
  • 不依赖人工标注,适合实际部署的高效推理场景。

新兴推理大模型如OpenAI-o1和DeepSeek-R1通过生成长链式思维(CoT)在复杂推理任务中表现优异,但过长的CoT导致推理延迟和内存消耗上升。现有长到短(Long2Short)方法常以牺牲准确性为代价降低长度,亟需兼顾效率与性能的方案。为此,我们提出TokenSqueeze,一种仅依赖模型自生成数据的新型长到短压缩方法。首先,根据问题复杂度自适应选择推理深度匹配的样本,避免过度压缩导致性能下降;其次,引入分布对齐的语言精炼机制,在不改变推理路径的前提下提升表达清晰度与简洁性。实验表明,该方法显著降低token用量并维持精度。值得注意的是,使用该方法微调后的DeepSeek-R1-Distill-Qwen-7B在MATH500基准上实现平均50%的token减少,同时保持原有准确率。整个流程无需人工标注短答案数据集,适用于多样化应用场景。代码已开源。

原文摘要 · Abstract (English)

Emerging reasoning LLMs such as OpenAI-o1 and DeepSeek-R1 have achieved strong performance on complex reasoning tasks by generating long chain-of-thought (CoT) traces. However, these long CoTs result in increased token usage, leading to higher inference latency and memory consumption. As a result, balancing accuracy and reasoning efficiency has become essential for deploying reasoning LLMs in practical applications. Existing long-to-short (Long2Short) methods aim to reduce inference length but often sacrifice accuracy, revealing a need for an approach that maintains performance while lowering token costs. To address this efficiency-accuracy tradeoff, we propose TokenSqueeze, a novel Long2Short method that condenses reasoning paths while preserving performance and relying exclusively on self-generated data. First, to prevent performance degradation caused by excessive compression of reasoning depth, we propose to select self-generated samples whose reasoning depth is adaptively matched to the complexity of the problem. To further optimize the linguistic expression without altering the underlying reasoning paths, we introduce a distribution-aligned linguistic refinement method that enhances the clarity and conciseness of the reasoning path while preserving its logical integrity. Comprehensive experimental results demonstrate the effectiveness of TokenSqueeze in reducing token usage while maintaining accuracy. Notably, DeepSeek-R1-Distill-Qwen-7B fine-tuned using our proposed method achieved a 50\% average token reduction while preserving accuracy on the MATH500 benchmark. TokenSqueeze exclusively utilizes the model's self-generated data, enabling efficient and high-fidelity reasoning without relying on manually curated short-answer datasets across diverse applications. Our code is available at https://github.com/zhangyx1122/TokenSqueeze.

大模型压缩推理优化自生成数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。