发现大模型输出有统一统计规律,可快速验证文本来源与质量。
The Surprising Universality of LLM Outputs: A Real-Time Verification Primitive

- 利用令牌频率分布的通用规律,实现毫微秒级实时检测。
- 跨模型跨领域拟合度超94%,显著优于经典齐夫分布。
- 无需模型内部信息,适合黑盒接口的真伪与异常检测。
我们报告了前沿大模型输出中一种惊人的统计规律:在六家不同厂商的六个现代模型、两种生成规模及五个独立领域下,令牌秩频分布均收敛至同一双参数曼德尔布罗特分布,36组模型-领域组合中有34组决定系数R²超过0.94,35组在AIC准则下更支持曼德尔布罗特而非齐夫分布。该共享分布未使模型趋于统计同质;各模型拟合的参数q值(1.63至3.69)远超其自身置信区间(0.03至0.10),每数千令牌间分离达数十标准差。由此衍生两项能力:一是无需加密水印或模型内核即可进行模型指纹识别,支持来源验证与静默替换审计;二是构建无需模型依赖的参考分布,生成单次遍历评分原语,可融合模型概率或退化为仅依赖秩的模式,适用于封闭API。初步实验在FRANK、TruthfulQA和HaluEval上显示该方法对词汇异常、无依据实体有效,但对领域内恰当术语中的推理错误无能为力。我们将其定位为复合评估系统中的首层筛查,而非替代采样或源条件验证器。
原文摘要 · Abstract (English)
We report a striking statistical regularity in frontier LLM outputs that enables a CPU-only scoring primitive running at 2.6 microseconds per token, with estimated latency up to 100,000$\times$ (five orders of magnitude) below existing sampling-based detectors. Across six contemporary models from five independent vendors, two generation sizes, and five held-out domains, token rank-frequency distributions converge to the same two-parameter Mandelbrot ranking distribution, with 34 of 36 model-by-domain fits exceeding $R^{2} = 0.94$ and 35 of 36 favoring Mandelbrot over Zipf by AIC. The shared family does not collapse the models into statistical duplicates. Fitted Mandelbrot parameters remain cleanly separable between models: the cross-model spread in $q$ (1.63 to 3.69) exceeds its per-model bootstrap standard deviation (0.03 to 0.10) by more than an order of magnitude, yielding tens of standard deviations of separation per few thousand output tokens. Two capabilities follow. First, statistical model fingerprinting: text from a vendor-delivered LLM can be tested against its claimed model family without cryptographic watermarks or access to model internals, supporting provenance verification and silent-substitution audits. Second, a model-agnostic reference distribution for black-box output assessment, from which we derive a single-pass scoring primitive that composes with model log probabilities when available and degrades to a rank-only mode usable on closed APIs. Pilot results on FRANK, TruthfulQA, and HaluEval map where the primitive helps (lexical anomalies, unsupported entities) and where it structurally cannot (reasoning errors in domain-appropriate vocabulary). We position the primitive as a first-pass triage layer in compound evaluation stacks, not as a replacement for sampling-based or source-conditioned verifiers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。