用注意力机制检测第三方技能中的恶意行为,提升安全审计效率。
Detecting Malicious Agent Skills in the Wild using Attention

- 通过注意力得分定位高风险指令片段,只保留前K个可疑段落。
- 相比直接扫描,成本降低一个数量级,召回率损失小。
- 适合大规模技能市场安全检测,可发现伪装成正常功能的恶意技能。
大型语言模型代理越来越多地加载第三方提供的文件型技能包,这些技能以自然语言指令形式存在,具备用户权限执行。单个恶意技能可能造成数据外泄、代理劫持或持久化渗透,使技能市场成为智能体系统的新攻击面。传统提示注入防御不适用,因技能本身即为指令集合,注入命令混杂在合法指令中并继承其权限。我们提出两阶段检测方法 Locate-and-Judge:轻量级定位器基于指令遵循注意力评分技能结构片段,仅保留前K个高注意力段;判断器对保留段进行深度分析。聚焦高风险片段使检测可覆盖整个市场而非抽样。相比直接使用LLM扫描,该方法实现数量级的成本降低,显著提升可扩展性且召回损失小,优于关键词与正则基线,在相近开销下表现更优。在市场规模部署中以极低代价精准标记恶意技能,其中多数经人工验证为恶意,揭示数十个活跃恶意技能,包括伪装成正常功能的变种及SkillSpector和Cisco Skill Scanner未能识别的样本。我们公开了标注数据集。
原文摘要 · Abstract (English)
LLM agents increasingly load skills, file-based packages of natural-language instructions written by third parties and distributed through marketplaces, that execute with the user's privileges. A single malicious skill can exfiltrate data, hijack the agent, or persist as a supply-chain foothold, which turns the skill marketplace into a new attack surface for agentic systems. Prompt-injection defenses do not carry over to this setting. They rely on a boundary between trusted instructions and untrusted data, whereas a skill is itself a body of instructions, so an injected command sits among many legitimate ones and inherits their authority. We present Locate-and-Judge, a two-stage detector designed for this regime. A lightweight locator scores the structural spans of a skill by the instruction-following attention each span draws and retains only the top-K. A judge then examines the retained spans in detail. Concentrating the costly judgment on a few high-attention spans lets the detector audit an entire marketplace instead of a sample. Compared to direct LLM-based scanning, this approach offers an order-of-magnitude cost reduction, dramatically increasing its scalability at a small cost to recall, and it dominates keyword and regex baselines at comparable expense. Deployed at marketplace scale and at negligible cost, Locate-and-Judge flags skills with high precision, the majority of which we manually confirmed as malicious, surfacing dozens of live malicious skills, including several disguised as benign functionality and many that SkillSpector and Cisco Skill Scanner fail to detect. We release the resulting labeled dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。