arXiv:2606.02981cs.CL2026-06

仅用一次采样就能预估大模型推理增强效果,省去昂贵验证开销。

Predicting Inference-Time Scaling Gains from Labeled Validation-Set Output Statistics

  • 基于单次采样统计,构建可稳定预测推理增益的紧凑特征集。
  • 在多模型、多任务下相关性达0.90,接近真实提升效果。
  • 适合快速筛选候选配置,避免全量奖励模型评分成本。

最佳-选-N推理(从语言模型生成N个候选答案,由奖励模型选出最优)能提升准确率,但其增益幅度因模型而异,现有方法需完整运行才能预测。先前工作利用采样输出与验证集正确性间的廉价统计特征(如样本一致性、多样性、置信度、正确答案位置等)关联模型行为,但未识别出稳定的、紧凑的预测因子。本文通过对单次标注验证集采样计算的特征拟合岭回归预测器,并使用自助-套索法进行特征集稳定性分析,结合显式线性近似残差的集中分析。在三个基础模型族、六种后训练方法及数学与推理任务域中,稳定性分析识别出一个严格的三特征核心:提示级一致度波动、标签辅助的首次正确样本位置、完成长度方差;基于此核心再加入熵项的紧凑岭预测器,在奖励模型验证下与实际最佳-选-N增益的斯皮尔曼相关系数达到ρ=0.90。该方法旨在通过标注验证集快速筛查候选配置,避免支付完整的奖励模型评分代价。

原文摘要 · Abstract (English)

Best-of-$N$ inference scaling (drawing $N$ candidate answers from a language model and returning the one a reward model ranks highest) improves accuracy by an amount that varies across models, but predicting that amount in advance currently requires running the procedure end-to-end. Prior work links cheap statistics of a model's sampled outputs and validation-set correctness (how often samples agree, how diverse they are, how confident the model is, and where correct samples appear) to model behavior, but does not isolate which of these form a stable, compact predictor of best-of-$N$ gain. We fit ridge predictors on features computed from a single labeled validation-set sampling pass, use bootstrap-Lasso as a stability analysis of the candidate feature set, and give a concentration analysis with an explicit linear-approximation residual. Across three base-model families, six post-training methods, and math and reasoning task domains, the stability analysis identifies a strict three-feature core spanning prompt-level agreement spread, label-assisted first-correct-sample position, and completion-length variance; a compact ridge predictor built from this core plus an entropy add-on reaches Spearman $ρ= 0.90$ with actual best-of-$N$ gain under a reward-model verifier. The intended use is labeled validation-set screening of candidate configurations before paying the full reward-model scoring cost.

推理优化预测模型奖励建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。