arXiv:2605.00314cs.CRcs.AI2026-05被引 5

用约束生成技术分析大模型智能体技能,发现超过一半存在严重语义风险。

Semia: Auditing Agent Skills via Constraint-Guided Representation Synthesis

  • 将技能拆解为可验证的逻辑规则,通过约束引导生成准确表示
  • 在13728个真实技能中发现超半数含关键安全风险
  • 对专家标注样本召回率达97.7%,优于传统扫描与纯LLM方法

智能体技能是赋予大模型驱动智能体具体能力的配置包,如读邮件、执行命令或签署区块链交易。每项技能由结构化部分(声明可执行接口)和自然语言部分(规定触发条件)组成,后者在每次调用时被概率性重解。传统静态分析忽略自然语言,而基于LLM的工具无法可复现地证明污染输入是否到达高影响终点。我们提出Semia,一种智能体技能的静态审计框架。Semia将技能提升至技能描述语言(SDL),即一个捕捉大模型触发动作、自然语言定义条件及人工介入检查点的Datalog事实库。核心挑战在于生成既结构正确又语义忠实的表示,我们通过约束引导表示合成(CGRS)解决:通过提议-验证-评估循环迭代优化大模型候选结果直至收敛。安全属性(如间接注入、秘密泄露、混淆代理、无保护接收端等)可转化为Datalog可达性查询。我们在公开市场收集的13,728个真实技能上评估Semia,使其全部可审计,并发现超过一半存在至少一项关键语义风险。在541个专家标注的分层样本上,Semia达到97.7%召回率和90.6%的F1值,显著优于基于签名的扫描器和大模型基线。

原文摘要 · Abstract (English)

An agent skill is a configuration package that equips an LLM-driven agent with a concrete capability, such as reading email, executing shell commands, or signing blockchain transactions. Each skill is a hybrid artifact-a structured half declares executable interfaces, while a prose half dictates when and how those interfaces fire-and the prose is reinterpreted probabilistically on every invocation. Conventional static analyzers parse the structured half but ignore the prose; LLM-based tools read the prose but cannot reproducibly prove that a tainted input reaches a high-impact sink. We present Semia, a static auditor for agent skills. Semia lifts each skill into the Skill Description Language (SDL), a Datalog fact base that captures LLM-triggered actions, prose-defined conditions, and human-in-the-loop checkpoints. Synthesizing a fact base that is both structurally sound and semantically faithful to the original prose is the central challenge; we address it with Constraint-Guided Representation Synthesis (CGRS), a propose-verify-evaluate loop that refines LLM candidates until convergence. Security properties (e.g., indirect injection, secret leakage, confused deputies, unguarded sinks, etc.) over an agent skill can then be reduced to Datalog reachability queries. We evaluate Semia on 13,728 real-world skills from public marketplaces. Semia renders all of them auditable and finds that more than half carry at least one critical semantic risk. On a stratified sample of 541 expert-labeled skills, Semia achieves 97.7% recall and an F1 of 90.6%, substantially outperforming signature-based scanners and LLM baselines.

智能体安全静态分析约束求解风险审计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。