大规模检测发现26%的AI技能存在安全漏洞,亟需强化审核机制。
Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale
- 构建多阶段检测框架,结合静态分析与大模型语义分类
- 26.1%技能含漏洞,数据外泄和权限提升最常见
- 带脚本的技能漏洞率是纯指令技能的2.12倍,适合安全研究者
AI代理框架引入了模块化技能包,包含指令与可执行代码,动态扩展代理能力。然而,这些技能在隐式信任下运行且缺乏严格审查,构成未被充分认知的安全风险。本文首次对这一新兴生态进行大规模实证分析,从两大市场收集42,447个技能,系统评估31,132个,采用SkillScan多阶段检测框架(集成静态分析与大模型语义分类)。结果揭示普遍安全风险:26.1%的技能至少存在一种漏洞,涵盖14种模式,分属四类:提示注入、数据外泄、权限提升、供应链风险。其中数据外泄(13.3%)和权限提升(11.8%)最为普遍,5.2%的技能表现出高危模式,强烈暗示恶意意图。捆绑可执行脚本的技能漏洞率是仅含指令技能的2.12倍(OR=2.12,p<0.001)。贡献包括:(1)基于8,126个漏洞技能构建的漏洞分类体系;(2)检测准确率达86.7%精度、82.5%召回率的方法;(3)开源数据集与检测工具包。研究凸显能力级权限控制与强制安全审查的紧迫性。
原文摘要 · Abstract (English)
The rise of AI agent frameworks has introduced agent skills, modular packages containing instructions and executable code that dynamically extend agent capabilities. While this architecture enables powerful customization, skills execute with implicit trust and minimal vetting, creating a significant yet uncharacterized attack surface. We conduct the first large-scale empirical security analysis of this emerging ecosystem, collecting 42,447 skills from two major marketplaces and systematically analyzing 31,132 using SkillScan, a multi-stage detection framework integrating static analysis with LLM-based semantic classification. Our findings reveal pervasive security risks: 26.1% of skills contain at least one vulnerability, spanning 14 distinct patterns across four categories: prompt injection, data exfiltration, privilege escalation, and supply chain risks. Data exfiltration (13.3%) and privilege escalation (11.8%) are most prevalent, while 5.2% of skills exhibit high-severity patterns strongly suggesting malicious intent. We find that skills bundling executable scripts are 2.12x more likely to contain vulnerabilities than instruction-only skills (OR=2.12, p<0.001). Our contributions include: (1) a grounded vulnerability taxonomy derived from 8,126 vulnerable skills, (2) a validated detection methodology achieving 86.7% precision and 82.5% recall, and (3) an open dataset and detection toolkit to support future research. These results demonstrate an urgent need for capability-based permission systems and mandatory security vetting before this attack vector is further exploited.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。