根据资源情况自动切换长短思维链，节省50%计算成本
Long or short CoT? Investigating Instance-level Switch of Large Reasoning Models
- 动态选择长或短思维链，按需分配计算资源
- 在有限预算下推理成本降低50%，准确率仍保持高位
- 适合对效率敏感的部署场景，如移动端或实时系统
随着大模型推理能力的提升，长思维链（CoT）提示在复杂任务上表现优异，但伴随显著的令牌消耗增加。本文通过全面实证分析比较了长、短两种CoT策略。结果表明，长CoT仅在生成预算充足时才带来边际收益，而在预算受限时，短CoT更具优势。为此，我们提出SwitchCoT框架，可自适应地在长与短思维链间切换，兼顾推理准确率与计算效率。该框架具备预算感知能力，适用于不同资源约束场景。实验显示，SwitchCoT可将推理成本降低最高达50%，且在资源受限条件下，性能甚至优于单独使用长或短CoT。
原文摘要 · Abstract (English)
With the rapid advancement of large reasoning models, long Chain-of-Thought (CoT) prompting has demonstrated strong performance on complex tasks. However, this often comes with a significant increase in token usage. In this paper, we conduct a comprehensive empirical analysis comparing long and short CoT strategies. Our findings reveal that while long CoT can lead to performance improvements, its benefits are often marginal relative to its significantly higher token consumption. Specifically, long CoT tends to outperform when ample generation budgets are available, whereas short CoT is more effective under tighter budget constraints. These insights underscore the need for a dynamic approach that selects the proper CoT strategy based on task context and resource availability. To address this, we propose SwitchCoT, an automatic framework that adaptively chooses between long and short CoT strategies to balance reasoning accuracy and computational efficiency. Moreover, SwitchCoT is designed to be budget-aware, making it broadly applicable across scenarios with varying resource constraints. Experimental results demonstrate that SwitchCoT can reduce inference costs by up to 50% while maintaining high accuracy. Notably, under limited token budgets, it achieves performance comparable to, or even exceeding, that of using either long or short CoT alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。