让大模型智能判断何时该多想、何时该省略，避免无效长思考。
Stable Adaptive Thinking via Advantage Shaping and Length-Aware Gradient Regulation
- 通过混合微调让模型学习思考与不思考的平衡策略。
- 在低复杂度任务上减少40%以上生成词数，准确率提升3.7点。
- 适合需要高效推理的部署场景，尤其对长尾问题更鲁棒。
大型推理模型（LRMs）通过延长推理链实现优异性能，但对低复杂度问题常出现过度思考。现有方法受限于准确率与效率的不稳定权衡及对异质推理行为的脆弱性。为此，我们提出一种两阶段稳定自适应思考框架。首先采用混合微调，使模型接触有思考与无思考的行为，建立良好初始化；随后通过保留正确性的优势塑造（CPAS）与长度感知梯度调节（LAGR）进行自适应强化学习，以避免抑制正确的长链推理，并在严重推理长度异质性下保持优化稳定。在Qwen2.5-1.5B和7B上的大量实验表明，相比强基线，本方法在准确率上提升最高达+3.7/+3.6点，同时生成词数减少40.6%/43.9%。跨不同难度问题及分布外任务的分析进一步验证了方法的鲁棒性与泛化能力。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) achieve strong performance through extended reasoning traces, but they often exhibit overthinking behavior for low-complexity queries. Existing efforts to mitigate this issue are fundamentally limited by unstable accuracy-efficiency trade-offs and poor robustness to heterogeneous reasoning behaviors. To address these challenges, we propose a two-stage framework for stable adaptive thinking in LRMs. The framework first applies Hybrid Fine-Tuning to expose the model to both thinking and no-thinking behaviors, establishing well-conditioned initialization. It then performs adaptive reinforcement learning with Correctness-Preserving Advantage Shaping (CPAS) to avoid suppressing correct long-chain reasoning, and Length-Aware Gradient Regulation (LAGR) to stabilize optimization under severe reasoning-length heterogeneity. Extensive experiments on Qwen2.5-1.5B and 7B show consistent improvements over strong baselines, achieving up to +3.7/+3.6 accuracy points while reducing generated tokens by 40.6%/43.9%. Further analyses across varying problem difficulties and out-of-distribution tasks confirm the robustness and generalization of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。