让大模型推理更高效：只重生成差的后半段，保留好的前半部分。
Selective Regenerative Decoding: Trajectory-Level Intervention for Inference-Time Reasoning
- 只重生成候选轨迹中差的部分，保留优质前缀，避免全丢浪费计算。
- 在多个数据集上，用更少的生成词数达到Best-of-N精度，效率提升1.28~1.36倍。
- 适合资源受限场景，尤其适合低算力下提升推理准确率的开发者。
推理时解码方法通过探索多个候选轨迹来提升大模型推理能力，但传统方法将每条轨迹视为原子单元：要么整体保留，要么彻底丢弃。这导致部分有潜力的候选轨迹因后半段质量差而被整体放弃，造成计算浪费。本文提出选择性再生解码（SRD），对每个候选轨迹进行分流处理：可保留、直接丢弃或仅重生成其质量较差的后缀部分，同时保留高质量前缀，无需更大目标模型。在合理假设下，SRD在样本效率上相比拒绝采样有1.28至1.36倍的提升，且期望轨迹质量更高，增益随候选池增大而增加。在MATH500、GPQA Diamond、HotpotQA和AlpacaEval等多个数据集上，使用多组生成-奖励模型组合，SRD以显著更少的生成词数达到与Best-of-N相当的准确率，并优于推测性拒绝法在低算力场景下的表现。通过实现段级干预而非整条轨迹选择，SRD开辟了推理时准确率-计算开销权衡的新区域。
原文摘要 · Abstract (English)
Inference-time decoding methods improve LLM reasoning by exploring multiple candidate trajectories, yet treat each trajectory as atomic: either retaining it whole or discarding it irreversibly. This wastes computation on partially promising candidates whose high-quality prefixes are abandoned alongside degraded suffixes. We introduce Selective Regenerative Decoding (SRD), which routes each candidate to discard, keep, or refine only the degraded portion of the suffix while preserving the useful prefix of borderline candidates, without requiring a larger target model. Under mild assumptions, SRD achieves a provable 1.28-to-1.36-fold gain in sample efficiency over rejection sampling with strictly higher expected trajectory quality, with the gain growing as the candidate pool grows. Across MATH500, GPQA Diamond, HotpotQA, and AlpacaEval with multiple generation-reward model pairs, SRD matches Best-of-N accuracy with substantially fewer generated tokens and outperforms speculative rejection in low-compute regimes. By enabling segment-level intervention rather than whole-trajectory selection, SRD opens a previously underexplored region of the accuracy-compute tradeoff for inference-time reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。