用可声明的规则框架，让AI代理安全调用外部工具。
ToolGuardian: Declarative Security for AI Agent-Tool Interactions
- 通过系统调用、模拟执行等逐步分析工具行为，生成结构化事实。
- 在16个工具上实现88%准确率，恶意工具识别F1达0.86。
- 适合关注AI代理安全、需可审计决策的开发者与研究者。
大型语言模型（LLM）代理越来越多依赖外部工具以扩展能力，但第三方工具在接口层面看似无害，其内部实现可能隐藏不安全行为。现有防御手段依赖弱元数据，将表征与策略判断合并为单一决策，或使用缺乏确定性与可审计性的启发式/大模型执行机制。本文提出ToolGuardian，一种基于策略的框架，通过预准入审查与任务感知的运行时授权保障代理-工具交互安全。ToolGuardian采用渐进式表征方法，将证据转化为结构化事实：描述反映声明意图，系统调用轨迹暴露粗粒度行为，模拟执行揭示可观测效果，源码分析识别潜在行为。核心贡献是基于答案集编程（ASP）的可声明策略层，显式推理能力、效果、任务上下文及多工具组合关系。我们使用相同输入与输出契约，对比了ASP与启发式及大模型实现的策略。在16个类MCP工具（含8个基于真实开源工具的恶意变体）和20个运行时场景中评估：预准入阶段，结合描述、系统调用与可观测效果证据，ASP达到0.86的拒绝类F1与88%准确率；运行时授权阶段，完整实现能正确分类所有场景，移除组合与合规规则后性能显著下降。
原文摘要 · Abstract (English)
LLM agents increasingly rely on external tools, expanding capability while creating a new security boundary: third-party tools may appear benign at the interface level while embedding unsafe behavior in implementation. Existing defenses rely on weak metadata, collapse characterization and policy judgment into a single decision, or use heuristic/LLM enforcement that lacks deterministic, auditable reasoning over task context and multi-tool composition. This paper presents ToolGuardian, a policy-driven framework for securing agent-tool interactions through pre-admission vetting and task-aware runtime authorization. ToolGuardian uses progressive characterization to convert evidence into structured facts: descriptions capture declared intent, system-call traces expose coarse behavior, mock execution reveals observed effects, and source analysis identifies latent behavior. ToolGuardian's core contribution is an Answer Set Programming (ASP)-based declarative policy layer that reasons explicitly over capabilities, effects, task context, and composition. We compare ASP against heuristic and LLM-based policy realizations using identical inputs and output contracts. We evaluate ToolGuardian on 16 MCP-style tools, including 8 malicious variants derived from real open-source tools, and 20 runtime scenarios. For vetting, ASP reaches a deny-class F1 of 0.86 and 88% accuracy using description, syscall, and observed-effect evidence. For runtime authorization, fully specified realizations classify all scenarios correctly, while ablations show that removing compositional and conformance rules substantially degrades performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。