用贝叶斯方法动态决定何时停止采样，省80%计算量仍保质量
BEACON: Bayesian Optimal Stopping for Efficient LLM Sampling
- 基于贝叶斯学习实时更新奖励分布信念，自适应判断是否继续采样
- 实验显示平均采样量减少80%，且保持与固定采样相当的输出质量
- 适合追求高效生成、需降低推理成本的研究者与工程落地场景
多次采样是提升大模型输出质量的常见方法，但会增加计算开销。核心挑战在于如何权衡准确率提升与效率损失，决定何时停止采样。为此，我们提出 BEACON（Bayesian Efficient Adaptive Criterion for Optimal N-stopping），一个基于贝叶斯学习的序贯搜索框架。BEACON 逐步生成策略模型响应，无需额外训练即可实时更新对奖励分布的后验信念，并通过权衡预期收益与计算成本来决定停止时机。当进一步探索的边际效用无法抵消成本时，采样终止。我们建立了理论最优性保证和实际可操作性，实证表明，BEACON 可将平均采样量减少高达 80%，同时维持高质量输出。我们还展示了其在低成本偏好数据生成中的应用潜力，并提出实用扩展方向，为未来研究提供可行动洞见。
原文摘要 · Abstract (English)
Sampling multiple responses is a common way to improve LLM output quality, but it comes at the cost of additional computation. The key challenge is deciding when to stop generating new samples to balance accuracy gains against efficiency. To address this, we introduce BEACON (Bayesian Efficient Adaptive Criterion for Optimal N-stopping), a principled adaptive sampling framework grounded in Sequential Search with Bayesian Learning. BEACON sequentially generates responses from the policy LLM, updates posterior belief over reward distributions in real time without further training, and determines when to stop by weighing expected gains against computational cost. Sampling terminates once the marginal utility of further exploration no longer justifies the expense. We establish both theoretical optimality guarantees and practical tractability, and show empirically that BEACON reduces average sampling by up to 80% while maintaining response quality. We further demonstrate BEACON's utility for cost-efficient preference data generation and outline practical extensions, offering actionable insights for future researchers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。