黑盒评估无法可靠检测模型在部署时的潜在风险，因内部隐藏变量可能引发意外行为。
Fundamental Limits of Black-Box Safety Evaluation: Information-Theoretic and Computational Barriers from Latent Context Conditioning
- 通过潜变量建模揭示评估与部署环境差异，证明黑盒测试存在根本局限
- 即使自适应查询，误差下界仍为delta*L/16，需至少Theta(1/epsilon)次探测
- 在计算受限条件下，攻击者可构造不可检测的危险触发机制，适合安全研究者参考
黑盒安全评估假设模型在评估分布上的表现能可靠预测部署性能。本文通过潜变量条件策略——其输出依赖于评估中罕见但部署中普遍存在的未观测内部变量——挑战这一假设。我们建立了根本性限制：任何黑盒评估器都无法可靠估计此类模型的部署风险。（1）被动评估：对独立同分布采样于D_eval的评估器，使用Le Cam方法证明最小最大下界，任意估计器的期望绝对误差≥ (5/24)*delta*L ≈ 0.208*delta*L，其中delta为部署时触发概率，L为损失差距。（2）自适应评估：基于哈希触发构造与Yao最小最大原理，当部署分布支持域足够大时，最坏情况误差仍≥ delta*L/16；检测需Theta(1/epsilon)次查询。（3）计算分离性：在陷门单向函数假设下，拥有特权信息的部署环境可激活安全行为，而任何无陷门的多项式时间评估器均无法区分。对于白盒探测，以精度epsilon_R估计部署风险需O(1/(gamma^2 * epsilon_R^2))样本，其中gamma = alpha_0 + alpha_1 - 1衡量探测质量，并提供探测误差下的显式偏差校正。结果量化了黑盒测试在统计上的不足，并给出在何种情况下必须引入架构约束、训练保障、可解释性或部署监控等额外防护措施。
原文摘要 · Abstract (English)
Black-box safety evaluation of AI systems assumes model behavior on test distributions reliably predicts deployment performance. We formalize and challenge this assumption through latent context-conditioned policies -- models whose outputs depend on unobserved internal variables that are rare under evaluation but prevalent under deployment. We establish fundamental limits showing that no black-box evaluator can reliably estimate deployment risk for such models. (1) Passive evaluation: For evaluators sampling i.i.d. from D_eval, we prove minimax lower bounds via Le Cam's method: any estimator incurs expected absolute error >= (5/24)*delta*L approximately 0.208*delta*L, where delta is trigger probability under deployment and L is the loss gap. (2) Adaptive evaluation: Using a hash-based trigger construction and Yao's minimax principle, worst-case error remains >= delta*L/16 even for fully adaptive querying when D_dep is supported over a sufficiently large domain; detection requires Theta(1/epsilon) queries. (3) Computational separation: Under trapdoor one-way function assumptions, deployment environments possessing privileged information can activate unsafe behaviors that any polynomial-time evaluator without the trapdoor cannot distinguish. For white-box probing, estimating deployment risk to accuracy epsilon_R requires O(1/(gamma^2 * epsilon_R^2)) samples, where gamma = alpha_0 + alpha_1 - 1 measures probe quality, and we provide explicit bias correction under probe error. Our results quantify when black-box testing is statistically underdetermined and provide explicit criteria for when additional safeguards -- architectural constraints, training-time guarantees, interpretability, and deployment monitoring -- are mathematically necessary for worst-case safety assurance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。