AI模型存在无法突破的性能天花板,可提前计算并用于设计更可靠的系统。
The Deterministic Horizon: Impossibility Results as Design Specifications for Trustworthy AI Systems
- 基于残差流容量不变性,提出可预计算的性能极限(确定性边界)
- 超过临界推理深度后,训练无法提升精度,准确率衰减呈超指数级
- 适用于可信AI设计,适合研究系统可靠性与边界约束的学者
大语言模型已能编写代码、起草法律文件和生成临床记录,但图灵、阿罗及无免费午餐定理等基本限制决定了计算的边界。本文将这些不可能性结果转化为设计规范。核心成果证明:仅由架构决定存在一个精度上限——超过临界推理深度后,无论适配器秩、样本量或损失函数如何,都无法提升性能。该确定性边界可在部署前通过层数与嵌入维度计算,12种Transformer架构中测得范围为19至31,对最优长度轨迹微调后最多恢复4个百分点。其机制源于残差流的容量不变性,信息论转换揭示了越过边界后的超指数级准确率衰减。同时,针对模幂运算提出了常数深度素数模电路的无条件电路复杂度下界。该论证还可迁移至多个子领域:任意模型误设下的偏好学习在样本复杂度上发生不连续跳跃;多阶段检索管道所需独立指标数不少于阶段数;标准真价拍卖在提示依赖估值的代理下失效;神经网络推理的零知识验证每非线性激活需付出110至190倍的开销。由此形成16条设计规范,每条包含可计算边界、量化违规代价与可构造设计规则:已证明两个组合关系,一个为诚实阻碍,四个仍开放。本文提出‘不可能性-规格化’方法论,供可信AI研究范式参考。所有人工智能的基本限制,都是设计规则。
原文摘要 · Abstract (English)
Large language models now write software, draft legal documents, and produce clinical notes, yet fundamental limits, from Turing and Arrow to the No Free Lunch theorems, shape what computation can do. This thesis turns such impossibility results from curiosities into design rules. Its flagship result proves an accuracy ceiling set by architecture alone: past a critical reasoning depth, no amount of training moves it, at any adapter rank, sample size, or loss function. Computable before deployment from layer count and embedding width, this Deterministic Horizon is measured between nineteen and thirty-one across twelve transformer architectures, and fine-tuning on optimal-length traces recovers under four percentage points. The mechanism is a capacity invariant of the residual stream, and an information-theoretic conversion yields super-exponential accuracy decay past the horizon. An unconditional circuit-complexity lower bound for modular exponentiation against constant-depth prime-modulus circuits complements this result. The same argument recasts across subfields: preference learning under any misspecified model jumps discontinuously in sample complexity; multi-stage retrieval pipelines require at least as many independent metrics as stages; standard truthful auctions fail for agents with prompt-dependent valuations; and zero-knowledge verification of neural inference pays a measured overhead of one hundred ten to one hundred ninety times per non-linear activation. Together these form a catalogue of sixteen specifications, each pairing a computable boundary, a quantified violation cost, and a constructive design rule: two compositions are proved, one pairing is an honest obstruction, and four remain open. The impossibility-specification methodology is offered for the generative research programme that trustworthy AI may need. Every fundamental limit of AI is also a design rule.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。