arXiv:2510.10168cs.AI2025-10被引 3

用拉格朗日优化实现大模型推理精简,兼顾准确率与效率。

Concise Reasoning in the Lens of Lagrangian Optimization

  • 将推理精简建模为约束优化问题,通过拉格朗日法转化为可解形式。
  • 输出长度减少65%,准确率提升15%,跨领域跨规模表现稳定。
  • 适合需要高效推理的场景,如低资源部署或实时问答系统。

大语言模型中的精简推理旨在仅生成到达最终答案所必需的中间步骤,以缓解过度思考问题。现有方法多依赖手工设计启发式规则,难以在简洁性与性能间取得平衡,且缺乏跨领域和模型规模的适应能力。本文提出一种原理严谨且实用的策略——性能感知长度更新(PALU)。PALU将精简推理建模为在性能约束下最小化响应长度的约束优化问题,并利用拉格朗日优化将其转化为可处理的无约束问题。作为实用方案,PALU通过三项近似简化复杂更新规则:(i) 使用离策略回放估计性能,(ii) 将拉格朗日乘子截断至两个极端值,(iii) 以分位数驱动的长度调整替代梯度更新。在DeepSeek-Distill-Qwen-1.5B上,平均覆盖五个基准测试,输出长度减少65%,准确率提升15%,优于多种对比方法。此外,PALU在逻辑、STEM和数学等不同领域,以及1.5B、7B、14B等多种模型规模下均表现出良好适应性,验证了其作为实用高效精简推理方法的潜力。

原文摘要 · Abstract (English)

Concise reasoning in large language models seeks to generate only essential intermediate steps needed to arrive at a final answer, thereby alleviating issues of overthinking. Most proposed approaches hinge on carefully hand-crafted heuristics, struggling to balance concision with performance, often failing to adapt across domains and model scales. In this work, we address these challenges by introducing a principled and pragmatic strategy, performance-aware length updating (PALU). As a principled algorithm, PALU formulates concise reasoning as a constrained optimization problem, minimizing response length subject to a performance constraint, and then applies Lagrangian optimization to convert it into a tractable unconstrained problem. As a pragmatic solution, PALU streamlines complicated update rules through three approximations: (i) estimating performance with off-policy rollouts, (ii) truncating the Lagrange multiplier to two extremes, and (iii) replacing gradient-based updates with quantile-driven length adjustments. PALU reduces output length by 65% while improving accuracy by 15% when applied to DeepSeek-Distill-Qwen-1.5B, averaged over five benchmarks, outperforming a range of alternative methods. Furthermore, PALU is demonstrated to adapt across both domain (logic, STEM and math) and model scale (1.5B, 7B, 14B) entrenching the algorithm as a practical and effective concise reasoning approach.

推理优化大模型拉格朗日

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。