自改进系统安全验证存在理论极限，无法同时保证无限收益与有限风险。
Information-Theoretic Limits of Safety Verification for Self-Improving Systems
- 用信息论建模安全门，证明分类器在重叠分布下无法实现无限收益
- 百万级修改中分类器最大收益仅约87，远低于验证器的50万
- 提出新型验证方法，可在零风险下保持正误判率，适用于大模型
自改进系统能否在累积风险可控的前提下实现无界收益？本文通过双条件形式化该问题：要求风险总和收敛（sum delta_n < infinity），而正确检出率总和发散（sum TPR_n = infinity）。我们证明，在幂律风险调度 delta_n = O(n^{-p})（p > 1）下，任何基于分类器的安全门在安全与不安全分布重叠时，均满足 TPR_n <= C_alpha * delta_n^beta，导致 sum TPR_n 必然收敛。该不可能性结果在指数上最优。另以NP计数法独立证明，可得更紧的13%边界。对于任意可求和的风险调度，分类器的最大可实现效用为 U*(N, B) = N * TPR_NP(B/N)，其增长速度为 exp(O(sqrt(log N)))，即次多项式。当 N=10^6、预算 B=1.0 时，分类器最大收益约87，而验证器可达约500,000。进一步证明，基于Lipschitz球的验证器可实现 delta=0 且 TPR>0，突破上述限制。对预归一化Transformer使用LoRA的范例给出了严格的Lipschitz约束，支持大模型规模验证。在GPT-2（d_LoRA=147,456）上验证：条件风险为0，正确检出率达0.352。完整实证分析见配套论文[D2]。
原文摘要 · Abstract (English)
Can a safety gate permit unbounded beneficial self-modification while maintaining bounded cumulative risk? We formalize this question through dual conditions -- requiring sum delta_n < infinity (bounded risk) and sum TPR_n = infinity (unbounded utility) -- and establish a theory of their (in)compatibility. Classification impossibility (Theorem 1): For power-law risk schedules delta_n = O(n^{-p}) with p > 1, any classifier-based gate under overlapping safe/unsafe distributions satisfies TPR_n <= C_alpha * delta_n^beta via Holder's inequality, forcing sum TPR_n < infinity. This impossibility is exponent-optimal (Theorem 3). A second independent proof via the NP counting method (Theorem 4) yields a 13% tighter bound without Holder's inequality. Universal finite-horizon ceiling (Theorem 5): For any summable risk schedule, the exact maximum achievable classifier utility is U*(N, B) = N * TPR_NP(B/N), growing as exp(O(sqrt(log N))) -- subpolynomial. At N = 10^6 with budget B = 1.0, a classifier extracts at most U* ~ 87 versus a verifier's ~500,000. Verification escape (Theorem 2): A Lipschitz ball verifier achieves delta = 0 with TPR > 0, escaping the impossibility. Formal Lipschitz bounds for pre-LayerNorm transformers under LoRA enable LLM-scale verification. The separation is strict. We validate on GPT-2 (d_LoRA = 147,456): conditional delta = 0 with TPR = 0.352. Comprehensive empirical validation is in the companion paper [D2].
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。