用零知识证明让AI代理执行可验证安全,防篡改、防劫持。
NiyamAI - An Intent-Bound AI Agent with Cryptographically Verifiable Guardrails using Zero-Knowledge Proofs
- 通过密码学承诺绑定操作意图,每步动作生成可验证证明
- 在2000个测试场景中实现88.8%准确率,误报率仅1%
- 第三方可独立验证安全性,适合高可靠性场景使用
具备工具调用能力的自主大模型代理易受提示注入、目标劫持和非法操作威胁。现有防护机制依赖运行环境内不可信的本地过滤器,无法向外部提供安全策略被正确评估的证据。本文提出NiyamAI,一种意图绑定的运行时防护架构,通过零知识证明实现执行完整性可验证。会话初始化时,允许工具与约束条件被哈希固化为不可变的意图合约。每次工具调用由确定性权威网关拦截,并由专用神经裁判(11→8→2前馈网络)分类。每个授权动作生成简洁zkSNARK证明,确保证策在已承诺合约下正确执行;仅当证明通过后才允许执行。在2000个AgentSafetyBench场景下,5折分层交叉验证且留出集评分,NiyamAI在1%误报率下达到88.8% F1(置信区间[85.5%, 92.1%]),显著优于Llama Prompt Guard 2(66.8%)、GPTOSSSafeguard(46.2%)和NeMo Guardrails(40.4%),McNemar检验均达p < 0.0001。每次批准动作需1.7秒生成证明,验证仅需51毫秒,证明大小18.6 KB,任何第三方均可无模型参数访问进行验证。此外,对NiyamAI自身机制进行18种对抗测试,发现并修复两个实现漏洞。
原文摘要 · Abstract (English)
Autonomous LLM agents with tool execution capabilities introduce severe security risks through prompt injection, goal hijacking, and unauthorized action invocation. Existing guardrails rely on unverified, host local software filters system prompts, semantic classifiers, policy engines that share the execution environment of the untrusted agent, offering no guarantee to an external observer that a safety policy was correctly evaluated. A compromised host produces no evidence of its own failure. This paper presents NiyamAI, an intent bound runtime guardrail architecture providing cryptographically verifiable execution integrity for autonomous agents. At session initialization, permitted tools and operational constraints are sealed into an immutable Intent Contract under a SHA256 commitment. Every tool invocation is intercepted by a deterministic authority gate and classified by a dedicated neural Judge (11->8->2 feedforward network). For each authorized action, NiyamAI generates a succinct zkSNARK proof certifying correct policy evaluation under the committed contract; execution proceeds only after that proof verifies. Across 2,000 AgentSafetyBench scenarios under 5fold stratified crossvalidation with out of fold scoring, NiyamAI achieves 88.8% F1 at a 1.0% false positive rate (bootstrap 95% CI [85.5%, 92.1%]), against 66.8% for Llama Prompt Guard 2, 46.2% for GPTOSSSafeguard, and 40.4% for NeMo Guardrails; McNemar's exact test confirms each margin at p < 0.0001. Proof generation adds 1.7 s per approved action, verification 51 ms, with an 18.6 KB proof verifiable by any third party without access to model parameters. We further subject NiyamAI's own enforcement mechanism to 18 adversarial vectors across six classes, disclosing two implementation vulnerabilities identified and remediated during development.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。