arXiv:2605.17609cs.LG2026-05被引 1

提出自适应生成-排序-验证方法,降低复杂验证成本。

Adaptive Generate-Rank-Verify: Inference-Time Search with Costly Verification

论文配图:Adaptive Generate-Rank-Verify: Inference-Time Search with Costly Verification
图 1 · 摘自论文原文
  • 动态调整生成与验证数量,根据奖励分数自适应搜索
  • 在数学推理和编程竞赛中,平均成本比固定策略低30%以上
  • 适合需要昂贵验证的场景,如代码执行、数学证明

许多推理时的语言模型流水线将廉价奖励信号与昂贵验证器结合,如数学推理中的精确答案检查或代码生成中的隐藏测试执行。本文从学习理论视角形式化该问题为生成式主动搜索:一种代价敏感的首次正例搜索,其中策略从未知分布中自适应采样候选,观察廉价评分,并付费获取验证标签直至找到正例。对于固定提示,生成器与奖励模型诱导出两个未知对象:奖励评分分布及条件成功函数。当两者已知时,我们通过动态规划刻画分布感知最优策略。在实际且现实的情形下,评分分布与成功函数均未知,我们提出ADAP——一种逐层自适应的生成-排序-验证算法,逐步增加采样响应数与顶部排名的验证数。在高分更可能通过验证的单调性假设下,我们证明ADAP的期望成本仅在常数因子内接近分布感知最优解。我们进一步基于中心星数给出学习理论下界,表明对评分-标签关系的结构假设是必要的。在数学推理与竞赛编程任务上的实验验证了其相较于固定非自适应策略及难度自适应基线的预期优势。

原文摘要 · Abstract (English)

Many inference-time language-model pipelines combine a cheap reward signal with an expensive verifier, such as exact answer checking in mathematical reasoning or hidden-test execution in code generation. We formalize this setting using a learning-theoretic lens as generative active search: a cost-sensitive first-positive search problem in which a policy adaptively samples candidates from an unknown distribution, observes cheap scores, and pays for verifier labels until it finds a positive example. For a fixed prompt, the generator and reward model induce two unknown objects: a distribution over reward scores and a score-conditioned success function. When these quantities are known, we characterize the distribution-aware optimal policy using a dynamic programming approach. In the realistic and practical setting where both the score distribution and success function are unknown, we propose ADAP, a shellwise adaptive generate-rank-verify algorithm that progressively increases the number of sampled responses and top-ranked verifications. Under the monotonicity assumption that higher reward scores are no less likely to pass verification, we show that ADAP achieves expected cost within a constant factor of the distribution-aware optimum. We complement this result with learning-theoretic lower bounds, based on a centered star number, showing that structural assumptions on the score--label relationship are necessary. Experiments on mathematical reasoning and competitive programming validate the predicted advantage over both fixed non-adaptive policies and difficulty-adaptive baselines.

推理优化自适应搜索验证成本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。