用前置推理路径压缩提升大模型推理效率，兼顾速度与准确率。
Can Reasoning Path still be Effective as Input? Bridging Post-Reasoning to Chain-of-Thought Compression
- 将推理路径作为上下文输入，简化大模型推理任务
- 在GSM8K上减少50%令牌使用，准确率还高出3.08%
- 适合追求高效推理的模型部署场景
近期研究通过长链式思维（CoT）实现了大语言模型（LLM）的高级推理，但以推理效率为代价。现有方法聚焦于压缩生成的CoT，损害了正确答案所需的必要信息。本文提出后置推理范式，将CoT作为上下文的一部分，降低模型推理复杂度。研究表明，后置推理显著缩短生成长度，其效果依赖于上下文CoT生成的效率与可靠性。为此，我们提出前沿推理路径（UCoT），一种高效的后置推理框架。UCoT训练轻量级压缩器以软标记形式提供上下文CoT，同时训练执行器（executor）利用该上下文进行最终答案生成。大量实验表明，UCoT在保持执行器强大推理能力的同时，大幅压缩了CoT长度。特别地，在Qwen2.5-7B-Instruct模型上应用时，GSM8K数据集的令牌使用量减少50%，性能优于当前最优方法（SOTA）3.08%。
原文摘要 · Abstract (English)
Recent developments have enabled advanced reasoning in Large Language Models (LLMs) via long Chain-of-Thought (CoT), trading efficiency during inference for performance. Existing works focus on compressing generated CoT in reasoning, which impairs the necessary information for deriving the correct answer. In this work, we propose post-reasoning, a reasoning paradigm that takes CoT as a part of context to simplify the reasoning task for LLMs. We find that post-reasoning significantly reduces the generation length of LLMs, but its effectiveness hinges on the efficiency and the reliability of the contextual CoT generation. Therefore, we propose Upfront CoT (UCoT), an efficient post-reasoning framework for CoT compression. UCoT trains a lightweight model (compressor) to provide contextual CoT in form of soft tokens and trains the LLM (executor) to leverage this contextual CoT for producing the final answer. Extensive experiments show that UCoT maintains the powerful reasoning ability of executor while significantly reducing the length of CoT. It is worth mentioning that when applying UCoT to the Qwen2.5-7B-Instruct model, the usage of tokens on GSM8K dataset is reduced by 50%, while the performance is 3.08% higher than that of the state-of-the-art (SOTA) method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。