提出无限可交换序列的集中性分析框架，用于精准量化AI基准测试的不确定性。
Bounded Difference Concentration for Infinitely Exchangeable Sequences with Applications to AI Benchmark Uncertainty
- 基于de Finetti测度条件化，分解函数偏差为抽样与隐混合波动
- 零和线性对比下隐混合项精确抵消，获得紧致无混合的Hoeffding型界
- 适用于多领域交叉的AI基准测试，无需假设分布即可低成本估算全集分数
我们研究无限可交换随机变量函数的集中性质。通过条件化de Finetti directing measure，证明具有有界差分常数 $c_1, \\.dots, c_n$ 的任意函数的偏离可分解为条件抽样波动与潜在混合波动。当该潜在混合服从 $σ_{\mathrm{mix}}^2$-次高斯分布时，建立有效方差代理为 $\frac{1}{4}\sum_i c_i^2 + σ_{\mathrm{mix}}^2$ 的集中不等式。关键发现是：对于零和线性对比（如子样本均值与全总体均值之差），潜在混合项恰好抵消。由此导出紧致、无混合的Hoeffding型界，直接呈现了近期有限可交换集中结果在无限可扩展极限下的de Finetti机制。我们将此框架应用于复合型AI基准测试（如MMLU），其中题目项在跨领域上天然具有可交换依赖。结果提供了一种领域分层的层次化模型，以界定准确率评分的不确定性，并给出一种无需分布假设、成本更低的统计保证，可从随机子集准确估计完整基准得分。
原文摘要 · Abstract (English)
We consider the concentration properties of functions of infinitely exchangeable random variables. By conditioning on the de Finetti directing measure, we show that the deviation of any function with bounded-difference constants $c_1, \dots, c_n$ decomposes into a conditional sampling fluctuation and a latent mixture fluctuation. When this latent mixture is $σ_{\mathrm{mix}}^2$-subgaussian, we establish a concentration inequality with an effective variance proxy of $\frac{1}{4}\sum_i c_i^2 + σ_{\mathrm{mix}}^2$. Crucially, we demonstrate that for zero-sum linear contrasts, such as the difference between a subsample mean and a full population mean, the latent mixture term cancels exactly. This cancellation yields a tight, mixture-free Hoeffding-type bound that provides a direct de Finetti mechanism for the infinite-extendibility limit of recent finite-exchangeable concentration results. We apply this framework to quantify uncertainty in composite AI benchmarks, such as MMLU, where question items naturally exhibit exchangeable dependence across domains. Our results provide both a domain-stratified hierarchical model for bounding the uncertainty of accuracy scores, and a distribution-free, cost-saving statistical guarantee for accurately estimating full benchmark scores from random subsets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。