arXiv:2605.04785cs.AIcs.CR2026-05被引 3

为AI代理工具调用提供实时安全拦截,防止误操作造成不可逆损失。

AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use

  • 在运行时拦截工具调用,结合去混淆与多步骤攻击链检测判断是否放行。
  • 在300个场景测试中达到95%判定准确率,630个对抗场景下仍保持96.7%准确率。
  • 适合需要高安全性部署的AI代理系统,尤其关注命令混淆与隐蔽攻击防护。

现代AI代理通过文件操作、命令行执行、HTTP请求和数据库查询等工具调用产生真实世界影响。一次不安全操作,如误删文件、凭据泄露或数据外泄,可能造成不可逆损害。现有防御手段存在缺陷:事后评估在执行后才衡量行为,静态规则无法识别混淆和多步上下文攻击,基础设施沙箱虽限制运行环境却缺乏对操作语义的理解。本文提出AgentTrust,一个运行时安全层,在工具调用执行前进行拦截,并返回结构化判断结果:允许、警告、阻止或需人工审查。AgentTrust融合了壳命令去混淆归一化、SafeFix安全替代建议、RiskChain多步攻击链检测以及缓存感知的LLM作为裁判模型以处理模糊输入。我们发布了涵盖六个风险类别的300个场景基准,以及额外独立构建的630个真实世界对抗场景。在内部基准上,仅使用生产规则集即可实现95.0%的判定准确率和73.7%的风险等级准确率,端到端延迟低于毫秒级。在630个场景基准上,经修补规则集评估(非零样本),仍达到96.7%的判定准确率,其中约93%针对壳命令混淆载荷有效。AgentTrust采用AGPL-3.0开源许可,并提供支持MCP协议的模型上下文服务端。

原文摘要 · Abstract (English)

Modern AI agents execute real-world side effects through tool calls such as file operations, shell commands, HTTP requests, and database queries. A single unsafe action, including accidental deletion, credential exposure, or data exfiltration, can cause irreversible harm. Existing defenses are incomplete: post-hoc benchmarks measure behavior after execution, static guardrails miss obfuscation and multi-step context, and infrastructure sandboxes constrain where code runs without understanding what an action means. We present AgentTrust, a runtime safety layer that intercepts agent tool calls before execution and returns a structured verdict: allow, warn, block, or review. AgentTrust combines a shell deobfuscation normalizer, SafeFix suggestions for safer alternatives, RiskChain detection for multi-step attack chains, and a cache-aware LLM-as-Judge for ambiguous inputs. We release a 300-scenario benchmark across six risk categories and an additional 630 independently constructed real-world adversarial scenarios. On the internal benchmark, the production-only ruleset achieves 95.0% verdict accuracy and 73.7% risk-level accuracy at low-millisecond end-to-end latency. On the 630-scenario benchmark, evaluated under a patched ruleset and not claimed as zero-shot, AgentTrust achieves 96.7% verdict accuracy, including about 93% on shell-obfuscated payloads. AgentTrust is released under the AGPL-3.0 license and provides a Model Context Protocol server for MCP-compatible agents.

AI安全运行时防护工具调用对抗攻击

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。