研究大模型在推理令牌受限时的表现，发现不完整推理会误导模型。
Broken Chains: The Cost of Incomplete Reasoning in LLMs
- 限定模型仅用代码、注释、两者混合或不推理，测试不同预算下的表现
- 代码推理比自然语言更抗压缩，50%预算下仍保持43-47%准确率
- 混合推理反而不如单一模式，且各模型鲁棒性差异显著
推理专用模型如GPT-5.1和DeepSeek-V3.2消耗大量推理计算资源于长链式思维（CoT）轨迹，但推理令牌成本高昂。不同推理模态（代码、自然语言、混合或无）在令牌约束下的表现如何？我们构建框架，将模型限制为仅通过代码、注释、两者兼用或不推理，并系统性地将令牌预算缩减至最优值的10%、30%、50%和70%。在数学基准（AIME、GSM8K、HMMT）上评估四个前沿模型（GPT-5.1、Gemini 3 Flash、DeepSeek-V3.2、Grok 4.1）。结果表明：（1）截断推理有害——DeepSeek-V3.2无推理时达53%，而50%预算下截断CoT仅17%；（2）代码推理渐进退化——Gemini注释性能归零，代码仍维持43-47%；（3）混合推理表现不佳，低于单一模态；（4）鲁棒性因模型而异——Grok在30%预算下保持80-90%，而OpenAI与DeepSeek降至7-27%。结果表明不完整推理会主动误导模型，对资源受限场景下的推理系统部署具有重要影响。
原文摘要 · Abstract (English)
Reasoning-specialized models like OpenAI's 5.1 and DeepSeek-V3.2 allocate substantial inference compute to extended chain-of-thought (CoT) traces, yet reasoning tokens incur significant costs. How do different reasoning modalities of code, natural language, hybrid, or none do perform under token constraints? We introduce a framework that constrains models to reason exclusively through code, comments, both, or neither, then systematically ablates token budgets to 10\%, 30\%, 50\%, and 70\% of optimal. We evaluate four frontier models (GPT-5.1, Gemini 3 Flash, DeepSeek-V3.2, Grok 4.1) across mathematical benchmarks (AIME, GSM8K, HMMT). Our findings reveal: (1) \textbf{truncated reasoning can hurt} as DeepSeek-V3.2 achieves 53\% with no reasoning but only 17\% with truncated CoT at 50\% budget; (2) \textbf{code degrades gracefully} as Gemini's comments collapse to 0\% while code maintains 43-47\%; (3) \textbf{hybrid reasoning underperforms} single modalities; (4) \textbf{robustness is model-dependent} as Grok maintains 80-90\% at 30\% budget where OpenAI and DeepSeek collapse to 7-27\%. These results suggest incomplete reasoning chains actively mislead models, with implications for deploying reasoning-specialized systems under resource constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。