用自然语言改进大模型的随机采样偏差
Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling
- 让大模型通过语言推理决定接受或拒绝样本
- 在伯努利分布上显著降低采样偏差
- 无需修改模型、适合需要可靠随机性的场景
大型语言模型（LLM）能准确描述概率分布，但在生成忠实样本方面仍表现不佳，限制了其在蒙特卡洛方法、基于代理的模拟和随机决策等任务中的应用。本文聚焦于伯努利分布，提出语言化拒绝采样（Verbalized Rejection Sampling, VRS），通过提示模型以自然语言推理并决定是否接受候选样本。尽管内部仍使用相同的伯努利机制，VRS在多个模型上显著降低了采样偏差。理论分析表明，在合理假设下，该方法优于直接采样，提升源于算法设计与提示结构的共同作用。结果表明，经典概率工具可通过自然语言嵌入到大模型工作流中，提升可靠性，且无需访问模型内部或进行复杂提示工程。
原文摘要 · Abstract (English)
Large language models (LLMs) can often accurately describe probability distributions using natural language, yet they still struggle to generate faithful samples from them. This mismatch limits their use in tasks requiring reliable stochasticity, such as Monte Carlo methods, agent-based simulations, and randomized decision-making. We investigate this gap between knowledge and sampling in the context of Bernoulli distributions. We introduce Verbalized Rejection Sampling (VRS), a natural-language adaptation of classical rejection sampling that prompts the LLM to reason about and accept or reject proposed samples. Despite relying on the same Bernoulli mechanism internally, VRS substantially reduces sampling bias across models. We provide theoretical analysis showing that, under mild assumptions, VRS improves over direct sampling, with gains attributable to both the algorithm and prompt design. More broadly, our results show how classical probabilistic tools can be verbalized and embedded into LLM workflows to improve reliability, without requiring access to model internals or heavy prompt engineering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。