用可验证的发布门禁,让自进化系统在扩展中保持安全不变。
Falsifiable Release Gates for Self-Improving Systems: Standing Invariants at Scale

- 每个新功能需通过预设的机器可检测试验,确保安全核心不变。
- 六次发布后,6个安全不变量始终成立,能力翻倍未引入新漏洞。
- 适用于需要高可信自进化系统的研发团队,如安全敏感型AI平台。
自进化智能体运行时的安全声明几乎总是自我评定:一份策略文件、一个护栏或README中的承诺。本文提出可验证的发布门禁机制,要求每一项新能力在发布前必须通过预先声明、机器可检的验收套件,同时保持一组固定的运行时不变量。我们在开源运行时Antahkarana中实现该方法,并持续追踪其演化过程,检验安全保证是否持续有效。关键安全属性——任何操作执行前必须获得由控制环签发的能力令牌——在有限状态模型中被完全机器检查;故意破坏模型后,检查器能生成最短反例,证明其有效性。随后经过六次发布,行动安全不变量INV-1至INV-6始终保持不变,且一次发布新增三项能力而无需新增不变量。在此严格检验下,又新增六个机器可检的保障族:可证明遗忘的内存、受控智能体、后量子记录下的校准拒绝、多子智能体协同框架、自进化循环本身,以及产物的驻留性。验收套件从122项增至563项。核心成果在于:在能力翻倍以上的情况下,安全内核未被削弱也未重构。最后这些组件首次在真实硬件上部署:门控自进化将小模型准确率从20%提升至70%,并自动拒绝仅增加置信度的候选方案;整个受控路径每请求耗时仅0.021毫秒,占模型推理时间的0.008%。我们公开运行时、工具和门禁套件,所有数据可通过单条命令复现。
原文摘要 · Abstract (English)
Safety claims for self-improving agent runtimes are almost always self-graded: a policy file, a guardrail, a promise in a README. We describe falsifiable release gates, a methodology in which every new capability must pass a pre-declared, machine-checkable acceptance suite before it ships, while a fixed set of standing invariants is preserved across every gate. We instantiate it in Antahkarana, an open runtime, then do what a method paper is only vindicated by: we follow the same runtime as it grows and ask whether the guarantees survive. The safety-critical property, that no action reaches an effector without a capability token minted by a control ring, is machine-checked exhaustively over the reachable states of a bounded model; a deliberately broken model yields the shortest counterexample, so the checker demonstrably has teeth. We then carry the runtime through six further releases. Across every one, the action-safety invariants INV-1 through INV-6 held without a single change, and one release added three capabilities while introducing no new invariant. Under the same teeth discipline, six more machine-checked families were added: memory with provable unlearning, a governed agent, calibrated abstention over a post-quantum record, a harness of many sub-agents, the self-improvement loop itself, and the residency of what it produces. The acceptance suite grew from 122 tests to 563. The load-bearing result sits in the negative space: across more than a doubling of capability, the safety core was neither weakened nor redesigned. The last families are the first on real hardware: gated self-improvement compounds a small model from 20% to 70% accuracy while auto-rejecting a candidate that only inflates confidence, and the whole governed path costs 0.021 ms per request, 0.008% of model inference. We release the runtime, tools, and gate suite; every number reproduces with a single command.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。