通过动态权重调整，让大模型推理更高效，少输出40%内容仍准确。
TL;DR: Too Long, Do Re-weighting for Efficient LLM Reasoning Compression
- 用动态权重平衡快思与慢思数据，自动删减冗余推理
- 在多个模型上减少近40%输出token，推理准确率不变
- 无需复杂标注，适合想压缩大模型推理的开发者
大型语言模型近年来借助强化学习和扩展链式思维（CoT）技术取得了显著进展。然而，如何在生成极长输出时实现高效的语言推理，已成为研究界关注的重点。本文提出一种基于动态比例的训练流程，无需依赖复杂的数据标注或多模型插值。通过持续平衡模型中系统1与系统2数据的权重，在保留推理能力的同时消除冗余推理过程。我们在DeepSeek-R1-Distill-7B和DeepSeek-R1-Distill-14B模型上，于多个难度各异的基准测试集上验证了该方法。结果表明，该方法可将输出词元数量减少近40%，同时保持推理准确性。代码与数据即将开源。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have recently achieved remarkable progress by leveraging Reinforcement Learning and extended Chain-of-Thought (CoT) techniques. However, the challenge of performing efficient language reasoning--especially during inference with extremely long outputs--has drawn increasing attention from the research community. In this work, we propose a dynamic ratio-based training pipeline that does not rely on sophisticated data annotations or interpolation between multiple models. We continuously balance the weights between the model's System-1 and System-2 data to eliminate redundant reasoning processes while preserving the model's reasoning capability. We validate our approach across models on DeepSeek-R1-Distill-7B and DeepSeek-R1-Distill-14B and on a diverse set of benchmarks with varying difficulty levels. Our method significantly reduces the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. Our code and data will be available soon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。