用五维静态分析检测恶意智能体技能，发现部分攻击仍难逃检测。
SkillsMetric: Mapping the Detection Boundary of Static Analysis for Malicious Agent Skills
- 构建五阶段分析框架，从模式密度到能力错配多维度评分。
- 对138K个技能检测达AUC 0.93，数据外泄与隐写攻击检出率93%。
- 发现常见命令的主机破坏和自然语言注入攻击几乎无法检测。
智能体技能——即增强基于大模型代理的结构化指令与脚本包——正快速普及，但其安全属性仍缺乏深入研究。本文提出 extsc{SkillsMetric}，一个五阶段静态分析框架，从模式密度、统计异常、数据流污染、导入异常和能力不匹配五个维度对技能包进行评分。我们构建了一个包含2,266个技能的对抗性评估数据集，覆盖代码级、系统级和语义级共16种攻击类型，并在完整的 SkillMD-138K 语料库上进行测试。该框架取得0.93的AUC和73.4%±0.5%的五折交叉验证F1值，对数据外泄（93%）和隐写载荷（93%）具有强检测能力。关键发现是：使用通用shell命令的主机破坏攻击在所有五阶段中均未被检测（0%），而通过自然语言操纵的提示注入攻击仅实现42%检测率。这些结果表明，仅靠静态分析不足以保障技能安全，亟需结合快速静态预筛与语义审查的纵深防御架构。
原文摘要 · Abstract (English)
Agent Skills---structured packages of instructions and scripts that augment LLM-based agents---are rapidly proliferating, yet their security properties remain under-explored. We present \textsc{SkillsMetric}, a five-stage static analysis framework that scores skill packages along pattern density, statistical anomaly, dataflow taint, import anomaly, and capability mismatch dimensions. We construct an adversarial evaluation dataset of 2{,}266 skills spanning 16~attack types across code-level, system-level, and semantic-level threats, and evaluate on the full SkillMD-138K corpus. Our framework achieves an AUC of 0.93 and 5-fold cross-validated F1 of 73.4\%$\pm$0.5\%, with strong detection of data exfiltration (93\%) and steganographic payloads (93\%). Crucially, we identify fundamental blind spots: \emph{host destruction} attacks using common shell commands evade all five stages (0\% detection), and \emph{prompt injection} via natural-language manipulation achieves only 42\% detection. These findings establish that static analysis alone is insufficient for skill security, motivating defense-in-depth architectures that combine fast static pre-screening with semantic review.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。