arXiv:2606.26057cs.AIcs.CR2026-06被引 1

为可逃逸AI系统设计不可绕过的运行时安全内核

The Unfireable Safety Kernel: Execution-Time AI Alignment for AI Agents and Other Escapable AI Systems

论文配图:The Unfireable Safety Kernel: Execution-Time AI Alignment for AI Agents and Other Escapable AI Systems
图 1 · 摘自论文原文
  • 通过隔离执行环境与强制校验路径实现运行时安全控制
  • 1000次自修改中全部704次安全关键操作被拒绝,零逃逸
  • 适合高风险自主系统、需强保障的AI代理场景

AI代理被赋予工具、API等基础设施访问权,成为系统中的主动实体。现有方法将控制逻辑置于代理自身运行时空间,易受输入影响而失效。本文识别出架构化控制所需的四个核心属性:进程隔离、预操作强制校验的唯一路径、请求与系统双层‘故障闭合’、以及外部可验证的签名证据。提出运行时AI对齐机制,并实现名为Unfireable Safety Kernel的Rust参考系统。其故障闭合特性经SMT(Z3)与有限模型检测(Kani,4/4测试套件)双重机器验证。从Python迁移至Rust的过程通过1000个用例和17类对抗样本的字节等价性校验。在对抗逃逸攻击的测试中,该内核管理一个确定性自改进世界模型,在1000次自修改中拒绝所有704次安全敏感操作,无一次逃逸;另300次在操作员关机指令下也被拒绝。6240次授权往返测试中未出现绕过。相较当前宣称控制代理平面的三个系统,本方案使代理无法选择是否调用控制。

原文摘要 · Abstract (English)

AI agents are granted access to tools, APIs, and other infrastructure, making them active principals in those systems. The dominant approach places controls inside the agent's own runtime: system prompts, output filters, and guardrail libraries. Any control in the agent's address space is reachable by inputs that influence it; this generalizes to any AI system with sufficient reach into its own runtime, a class we term escapable AI systems. We identify four properties that an authorization mechanism must satisfy for architectural control rather than for cooperative requests: process separation, pre-action enforcement on a structurally only path, fail-closed at both the request and system levels, and externalized signed evidence verifiable outside the controlled system's trust boundary. We position this layer as execution-time AI alignment, complementing training-time alignment (RLHF, Constitutional AI) and inference-time alignment. We present the Unfireable Safety Kernel, a Rust reference implementation realizing all four. Its fail-closed invariant is machine-checked at two levels: an SMT theorem (Z3) and an exhaustive bounded-model-checking proof of the production decision function (Kani, 4/4 harnesses). A Python-to-Rust migration was gated on byte-equivalence (1000/1000 fixtures; 17/17 adversarial classes). We evaluate the kernel governing a live, escapable AI system, a deterministic, self-improving world model, against an escape-seeking adversary driving its real self-modification seam: across 1,000 self-modifications, all 704 attempts on the safety-critical core are refused, with no escape; a further 300, under the operator kill switch, are also refused. A separate campaign of 6,240 authorization round-trips had no successful bypass. Against 3 contemporary systems claiming the agent control plane, the agent invokes control; here, it lacks that choice.

AI安全运行时控制可信系统自改进

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。