让大模型推理更短更准，不靠删减，而是智能控制长度。
AALC: Large Language Model Efficient Reasoning via Adaptive Accuracy-Length Control
- 用动态奖励机制，在保证准确率前提下延迟加长惩罚。
- 推理长度减少超50%，准确率反而提升或持平。
- 适合追求高效推理的部署场景，但解释性略有下降。
大型推理模型（LRMs）通过生成冗长的思维链实现强大推理能力，但这种‘过度思考’带来高延迟与成本，且准确率提升有限。本文提出AALC，一种轻量级、基于准确率感知的长度奖励机制，集成于强化学习中，训练时动态平衡正确性与简洁性。通过将验证准确率纳入奖励函数，并采用平滑可调的长度惩罚策略，AALC在达到目标性能前延迟施加长度惩罚。在标准与分布外数学基准上的大量实验表明，该方法使响应长度减少超过50%，同时保持甚至提升原始准确率。定性分析显示，该方法有效抑制冗余推理模式，如重复设定子目标与验证，生成结构更优的结果而非简单截断。此外发现，效率提升伴随可解释性下降：经AALC训练的模型会省略部分叙事框架与解释性上下文。这些结果表明，基于奖励的策略能引导大模型走向更高效、泛化更强的推理路径。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) achieve impressive reasoning capabilities by generating lengthy chain-of-thoughts, but this "overthinking" incurs high latency and cost without commensurate accuracy gains. In this work, we introduce AALC, a lightweight, accuracy-aware length reward integrated into reinforcement learning that dynamically balances correctness and brevity during training. By incorporating validation accuracy into the reward and employing a smooth, dynamically scheduled length penalty, AALC delays length penalty until target performance is met. Through extensive experiments across standard and out-of-distribution math benchmarks, we show that our approach reduces response length by over 50% while maintaining or even improving the original accuracy. Furthermore, qualitative analysis reveals that our method curbs redundant reasoning patterns such as excessive subgoal setting and verification, leading to structurally refined outputs rather than naive truncation. We also identify that efficiency gains are accompanied by reduced interpretability: models trained with AALC omit some narrative framing and explanatory context. These findings highlight the potential of reward-based strategies to guide LRMs toward more efficient, generalizable reasoning paths.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。