arXiv:2512.01457cs.LGcs.AI2025-12被引 7

让大模型实时评估推理成本与收益,自动决定何时停止或重试。

Zero-Overhead Introspection for Adaptive Test-Time Compute

  • 利用原有计算结果预测奖励与剩余开销,无需额外模型或延迟。
  • 在数学题上比投票法准确率高12%,且平均成本更低或相当。
  • 适合需要高效推理的场景,如实时问答、资源受限部署。

大型语言模型在推理中表现出色,但缺乏自我评估能力,无法预判自身成功概率及所需计算量。人类会根据实时判断调整努力程度、尝试次数或终止决策,而模型缺乏此类元认知能力。现有的测试时扩展方法(如Best-of-N)因固定采样预算导致成本和延迟上升,且缺乏置信度信号会误导用户、阻碍工具升级并降低可信度。已有方法如学习型验证器或奖励模型可提供置信度,但需额外模型或前向传播,带来显著开销。本文提出ZIP-RC,通过复用同一前向传播中的预留或空闲logits,实时输出最终奖励与剩余生成长度的联合分布,实现零开销的自省预测。该联合分布用于计算采样效用——即预期最大奖励、总计算量与延迟的线性组合。推理过程中,通过最大化效用选择继续生成或启动新采样。在混合难度数学基准测试中,ZIP-RC相较多数投票法提升准确率最高达12%,且平均成本相等或更低,并呈现平滑的品质-计算-延迟帕累托前沿。通过提供实时奖励-成本自省,ZIP-RC实现了自适应、高效的推理。

原文摘要 · Abstract (English)

Large language models excel at reasoning but lack key aspects of introspection, including anticipating their own success and the computation required to achieve it. Humans use real-time introspection to decide how much effort to invest, when to make multiple attempts, when to stop, and when to signal success or failure. Without this, LLMs struggle to make intelligent meta-cognition decisions. Test-time scaling methods like Best-of-N drive up cost and latency by using a fixed budget of samples regardless of the marginal benefit of each one at any point in generation, and the absence of confidence signals can mislead people, prevent appropriate escalation to better tools, and undermine trustworthiness. Learned verifiers or reward models can provide confidence estimates, but do not enable adaptive inference and add substantial cost by requiring extra models or forward passes. We present ZIP-RC, which equips models with zero-overhead introspective predictions of reward and cost. At every token, ZIP-RC reuses reserved or unused logits in the same forward pass as next-token prediction to output a joint distribution over final reward and remaining length -- no extra models, architecture change, or inference overhead. This full joint distribution is used to compute a sampling utility which is the linear combination of the expected maximum reward, total compute, and latency of set of samples if generated to completion. During inference, we maximize this utility with meta-actions that determine which prefix of tokens to continue or initiate sampling from. On mixed-difficulty mathematical benchmarks, ZIP-RC improves accuracy by up to 12% over majority voting at equal or lower average cost, and traces smooth Pareto frontiers between quality, compute, and latency. By providing real-time reward-cost introspection, ZIP-RC enables adaptive, efficient reasoning.

自省推理测试时扩展效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。