动态调节大模型推理中的随机性与计算量,提升效率与准确率
AutoCRAT: Within-trajectory Joint Control of Stochasticity and Compute for LLM Reasoning

- 在生成过程中联合调控采样随机性与推理时长
- 平均减少40%以上推理令牌消耗,准确率提升1.5%-4.5%
- 适合追求高效推理的部署场景与多任务应用
大语言模型(LLMs)的强推理能力依赖于推理阶段的决策,但现有方法多采用静态、统一策略,难以适应不同任务与推理阶段。尽管已有自适应方法,但通常仅独立调整采样随机性或推理计算量,未建模二者在单次推理轨迹中的协同作用。为此,本文提出AutoCRAT,一种面向冻结主干模型的解码端控制器,基于解码时可获取信号,联合调整采样随机性与推理预算。该方法在离散动作空间中运行,仅在语义边界更新控制决策,兼顾稳定性与响应性。在6个基准上的综合评估表明:(I) 相比推荐的静态配置,平均节省13.8%-52.7%的推理令牌;(II) 在相对准确率上超越推荐静态与自适应基线1.5%-4.5%;(III) 具有优异的跨模型迁移能力。
原文摘要 · Abstract (English)
Large language models (LLMs) achieve strong reasoning performance, which depends critically on inference-time decisions. Yet these decisions are commonly handled by static, one-size-fits-all policies, limiting adaptation to diverse tasks and reasoning stages. Recent adaptive methods partially address this limitation, but they primarily adapt either decoding stochasticity (how the model explores) or reasoning compute (how long the model reasons) in isolation, leaving their interaction within a single reasoning trajectory unmodeled. To address this challenge, we shift toward a within-trajectory joint control view, and instantiate it in AutoCRAT, a decoder-side controller for frozen backbones. Using only signals available during decoding, AutoCRAT jointly adjusts sampling stochasticity and reasoning budget during generation. AutoCRAT operates over a discrete action space and updates control decisions only at semantic boundaries, improving stability while remaining responsive to the evolving reasoning process. Comprehensive evaluation across 6 benchmarks demonstrates that AutoCRAT (I) uses 13.8-52.7% fewer inference tokens on average than recommended static configurations, (II) surpasses recommended static and adaptive baselines by 1.5-4.5% in relative accuracy, and (III) enjoys strong cross-backbone transferability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。