让大模型推理不依赖输出token，降低计算成本。
TwT: Thinking without Tokens by Habitual Reasoning Distillation with Multi-Teachers' Guidance
- 通过多教师引导压缩推理过程，将思考内化为习惯行为。
- 减少输出token数，推理准确率提升13.6%。
- 适合无监督场景，部署效率高，适合实际应用。
大型语言模型在问题求解中通过引入推理过程取得了显著进展，但这也导致推理阶段输出的token数量增加，带来更高的计算开销。为此，我们提出TwT（Thinking without Tokens），一种基于多教师引导的习惯性推理蒸馏方法，在保持高性能的同时降低推理时开销。该方法引入习惯性推理蒸馏机制，通过受人类认知启发的教师引导压缩策略，将显式推理过程内化为模型的惯性行为。同时，提出双准则拒绝采样（DCRS）技术，利用多个教师模型生成高质量且多样化的蒸馏数据集，使方法适用于无监督场景。实验表明，TwT在显著降低推理成本的同时保持优异性能，相比其他蒸馏方法，在更少输出token下实现最高13.6%的准确率提升，为高效LLM部署提供了实用解决方案。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have made significant strides in problem-solving by incorporating reasoning processes. However, this enhanced reasoning capability results in an increased number of output tokens during inference, leading to higher computational costs. To address this challenge, we propose TwT (Thinking without Tokens), a method that reduces inference-time costs through habitual reasoning distillation with multi-teachers' guidance, while maintaining high performance. Our approach introduces a Habitual Reasoning Distillation method, which internalizes explicit reasoning into the model's habitual behavior through a Teacher-Guided compression strategy inspired by human cognition. Additionally, we propose Dual-Criteria Rejection Sampling (DCRS), a technique that generates a high-quality and diverse distillation dataset using multiple teacher models, making our method suitable for unsupervised scenarios. Experimental results demonstrate that TwT effectively reduces inference costs while preserving superior performance, achieving up to a 13.6% improvement in accuracy with fewer output tokens compared to other distillation methods, offering a highly practical solution for efficient LLM deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。