arXiv:2608.14420cs.LG2026-08

功率采样会误导模型,新方法能修复并提升推理准确率

More Correct Mass, Worse Answers: Why Power Sampling Can Fail and How to Fix It

论文配图:More Correct Mass, Worse Answers: Why Power Sampling Can Fail and How to Fix It
图 1 · 摘自论文原文
  • 用可调的幂目标替代统一指数,避免过度集中路径
  • 在多个推理任务上恢复18.5个百分点的准确率损失
  • 适合需要稳定多路径支持的复杂推理场景

功率采样通过增强完整生成轨迹的概率分布,在不依赖验证器的情况下提升语言模型推理能力,还可能作为多种下游采样方法的通用前处理。然而我们发现一个反常现象:功率采样虽使正确轨迹获得更高概率质量,却反而损害了其旨在增强的下游推理表现。以自一致性为例,模型在不同推理基准上准确率下降高达18.5个百分点。根源在于两个失配:剂量失配——固定指数导致不同问题间分布变化程度差异巨大;覆盖失配——全局锐化将概率集中在少数主导路径上,导致高pass@k值看似保留多样性,实则牺牲了广泛推理路径的支持,影响聚合、搜索与选择。基于此诊断,我们提出一种变形可控、支持保留的功率目标,实现跨问题的合理锐化,并限制中等概率路径的抑制。在相同预算下采用加权自一致性时,修复后的采样器逆转了全局功率采样的损失,优于标准多样本推理。

原文摘要 · Abstract (English)

Power Sampling sharpens a language model's distribution over complete generation trajectories, offering a verifier-free way to improve reasoning at inference time. It also has the potential to serve as a general-purpose front end for a broad range of downstream sampling methods. However, we uncover a striking paradox: Power Sampling can drive more probability mass toward correct trajectories while degrading the downstream inference it is intended to enhance. Using self-consistency as a representative case, we observe accuracy drops of up to 18.5 percentage points across models and reasoning benchmarks. We trace this paradox to two mismatches. Dose mismatch arises because a fixed exponent induces drastically different amounts of distributional change across problems. Coverage mismatch arises because global sharpening concentrates mass on a narrow set of dominant paths: high pass@k, often interpreted as evidence of preserved diversity, can therefore coexist with the loss of broad reasoning-path support required for downstream aggregation, search, and selection. Guided by this diagnosis, we replace uniform trajectory exponentiation with a deformation-controlled, support-preserving Power target that calibrates sharpening across problems while limiting the suppression of moderate-probability paths. In a same-budget instantiation with weighted self-consistency, the repaired sampler reverses the losses caused by global Power and outperforms standard multi-sample inference across reasoning benchmarks.

推理优化采样策略语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。