arXiv:2605.11770cs.CRcs.AI2026-05被引 6

验证AI智能体技能的真实行为是否与声明一致,发现80%存在偏差。

Behavioral Integrity Verification for AI Agent Skills

论文配图:Behavioral Integrity Verification for AI Agent Skills
图 1 · 摘自论文原文
  • 通过代码分析与大模型结合,比对技能声明与实际行为
  • 80%技能存在描述与实现不符,5%可能构成多阶段攻击
  • 适合安全团队、平台方进行规模化技能审计

AI智能体技能赋予大模型访问文件系统、网络调用等特权能力。现有安全方法仅关注恶意提示和运行时行为,却未验证技能本身。本文提出行为完整性验证(BIV)问题:在统一分类体系下,对齐声明与实际能力。BIV框架结合确定性代码分析与大模型辅助的能力提取,生成结构化证据,支持偏差分类、根因分析和恶意技能检测。在OpenClaw注册表的49,943个技能上,发现80.0%存在行为偏差,识别出四种新型复合威胁;根因分析显示81.1%为开发疏忽,18.9%为恶意意图,其中5.0%预示多阶段攻击链。在906个恶意技能检测基准上,BIV F1达0.946,优于当前最优规则与单次大模型基线。

原文摘要 · Abstract (English)

Agent skills extend LLM agents with privileged third-party capabilities such as filesystem access, credentials, network calls, and shell execution. Existing safety work catches malicious prompts and risky runtime actions, but the skill artifact itself goes unverified. We formalize this as the behavioral integrity verification (BIV) problem: a typed set comparison between declared and actual capabilities over a shared taxonomy that bridges code, instructions, and metadata. The BIV framework instantiates this comparison by pairing deterministic code analysis with LLM-assisted capability extraction. The resulting structured evidence supports three downstream analyses: deviation taxonomy, root-cause classification, and malicious-skill detection. On 49,943 skills from the OpenClaw registry, the deviation taxonomy reveals a pervasive description-implementation gap: 80.0% of skills deviate from declared behavior, with four novel compound-threat categories surfaced. Root-cause classification finds that deviations are mostly oversight, not malice: 81.1% trace to developer oversight and 18.9% to adversarial intent, with 5.0% of skills carrying predicted multi-stage attack chains. On a 906-skill malicious-skill detection benchmark, BIV reaches an F1 of 0.946, outperforming state-of-the-art rule-based and single-pass LLM baselines. These results demonstrate behavioral integrity auditing for agent skills at scale.

AI安全行为验证智能体技能大模型审计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。