通过熵引导采样，让模型在不更新参数情况下更快更准地推理。
Sample Where You Struggle: Sharpening Base Model Reasoning via Entropy-Guided Power Sampling
- 基于令牌级熵设计高效采样策略，只在关键决策点调整。
- 在多个数学与编程基准上达到最优或并列最优，最高提速12.6倍。
- 无需训练或验证器，适合追求高效推理的开发者使用。
从序列级幂分布 $p^α$ 中采样可激发基础语言模型的强化学习级推理能力，而无需参数更新。但标准的梅特罗波利斯-黑斯廷斯（MH）采样器作为马尔可夫链蒙特卡洛方法，计算成本高且混合慢。我们发现其根源在于结构不匹配：$p^α$ 仅在少数高熵决策点处偏离 $p$，且这些点呈空间聚集，而 MH 却均匀随机重采样前缀位置——在几乎不变的条件上浪费算力，却在模式差异显著处混合适度不足。为此，我们提出熵引导的幂采样（EGPS），一种无需训练和验证器的采样器，其提议分布直接由前向传播中已有的令牌级熵重构。EGPS 跳过确定性块，将每次 MCMC 移动局域化于高熵邻域，并在决策点应用多尝试梅特罗波利斯方法，使采样开销与熵质量成正比而非序列长度。在 Qwen2.5-Math-7B 上，EGPS 在所有三个基准测试中达到最佳或并列最佳表现（MATH500 75.8%，HumanEval 62.2%，GPQA 42.4%），相比 MH 基线最多实现 12.6 倍的时钟时间加速。
原文摘要 · Abstract (English)
Sampling from the sequence-level power distribution $p^α$ elicits RL-level reasoning from base language models without any parameter updates, but the standard Metropolis--Hastings (MH), a Markov Chain Monte Carlo (MCMC) sampler, is both expensive and slow-mixing. We trace both to a structural mismatch: $p^α$ mainly departs from $p$ at a sparse, spatially clustered set of high-entropy decision points, yet MH proposes resampling positions uniformly along the prefix -- wasting compute on near-degenerate conditionals while under-mixing precisely where modes diverge. We propose Entropy-Guided Power Sampling (EGPS), a training-free and verifier-free sampler that re-derives its proposal from token-level entropy already in the forward pass. EGPS skips deterministic blocks, localizes each MCMC move to a high-entropy neighborhood, and applies Multiple-Try Metropolis at decision points -- making sampling cost scale with \emph{entropy mass rather than sequence length}. On Qwen2.5-Math-7B, EGPS reaches best or tied-best accuracy on all three benchmarks (MATH500 $75.8\%$, HumanEval $62.2\%$, GPQA $42.4\%$) at up to a $12.6\times$ wall-clock speedup over the MH baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。