arXiv:2601.05414cs.CLcs.AI2026-01ACL被引 5

大模型生成随机数能力差,批量生成效果更差

Large Language Models Are Bad Dice Players: LLMs Struggle to Generate Random Numbers from Statistical Distributions

  • 对比批量生成与独立请求两种方式,发现生成效果差异巨大
  • 11个模型中仅7%在批量生成中通过统计检验,10个在独立请求中全失败
  • 模型在复杂分布和长序列生成中偏差加剧,不适合需统计保证的任务

随着大语言模型从对话接口转向随机流程与接近通用智能系统的组成部分,忠实采样指定概率分布的能力已从理论兴趣变为功能需求。我们对前沿大模型进行了首次大规模、基于统计的原生概率采样审计,评估了11个模型在15种分布上的表现。为区分失效模式,采用双协议设计:批量生成(单次响应生成1000个样本)与独立请求(1000次无状态调用)。结果显示显著协议不对称:批量生成的统计有效性仅达7%中位通过率,而独立请求几乎全部崩溃,11个模型中有10个在所有分布上均未通过。此外,采样精度随分布复杂度上升而单调下降,且随采样规模N增大而恶化。最后,我们证明这些缺陷会传播至下游任务,导致多选题生成中无法保持均匀答案位置,文本到图像提示生成中系统性违反人口统计目标。结果表明当前大模型缺乏功能性内部采样器,涉及统计保证的应用必须依赖外部工具。

原文摘要 · Abstract (English)

As large language models (LLMs) transition from chat interfaces to integral components of stochastic pipelines and systems approaching general intelligence, the ability to faithfully sample from specified probability distributions has become a functional requirement rather than a theoretical curiosity. We present the first large-scale, statistically powered audit of native probabilistic sampling in frontier LLMs, benchmarking 11 models across 15 distributions. To disentangle failure modes, we employ a dual-protocol design: Batch Generation, where a model produces $N{=}1000$ samples within one response, and Independent Requests, comprising $N{=}1000$ stateless calls. We observe a sharp protocol asymmetry: batch generation achieves only modest statistical validity, with a 7% median pass rate, while independent requests collapse almost entirely, with 10 of 11 models passing none of the distributions. Beyond this asymmetry, we reveal that sampling fidelity degrades monotonically with distributional complexity and aggravates as the sampling horizon $N$ increases. Finally, we demonstrate how the propagation of these failures into downstream real-world application tasks introduces systematic biases: models fail to enforce uniform answer-position constraints in Multiple Choice Question generation and systematically violate demographic targets in attribute-constrained text-to-image prompt synthesis. These findings indicate that current LLMs lack a functional internal sampler, necessitating external tools for applications requiring statistical guarantees.

大模型随机采样统计偏差生成质量

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。