提出分层筛选框架，精准识别恶意AI技能的多模态威胁。
SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skills
- 三层递进式检测：正则、代码抽象语法树、元数据与LLM并行分析
- 在49,592个真实技能上实现F1=0.929，每项检测成本仅0.006美元
- 可部署为飞书机器人，支持上下文重判，适合安全审计与平台防护
AI代理技能融合自然语言指令与可执行代码，并继承代理的文件系统、凭证和网络访问权限。攻击可同时跨越文本与代码，而传统正则或纯代码分析仅覆盖单一模态。SkillSieve采用三层递进式筛选：面向召回的正则、抽象语法树（AST）与元数据分析；四项并行的LLM安全子任务；以及独立三模型仲裁机制，对分歧进行辩论。我们在49,592个真实ClawHub技能、390个标注基准数据集及100个对抗样本上评估，涵盖五种逃避技术，在440美元的ARM设备上运行。完整流程实现F1=0.929（精确率0.912，召回率0.945），平均单次检测成本0.006美元。可选的XGBoost快速路径减少32%的第二、三层调用，仅导致F1下降1.7点但保持召回率。在52个飞书/飞书包中，第二层通过上下文分析将14个第一层误报中的13个重新判定为安全；系统已作为飞书聊天机器人部署。代码、标签与汇总结果开源。
原文摘要 · Abstract (English)
Agent skills combine natural-language instructions with executable code while inheriting an agent's filesystem, credential, and network access. Attacks can span prose and files, whereas regex and code-only analyzers cover only one modality. SkillSieve applies three progressively deeper layers: recall-oriented regex, AST, and metadata triage; four parallel LLM security sub-tasks; and an independent three-model jury with debate on disagreement. We evaluate 49,592 real ClawHub skills, a 390-skill labeled benchmark, and 100 adversarial samples across five evasion techniques on a 440 USD ARM board. The full pipeline achieves F1 = 0.929 (precision 0.912, recall 0.945) at an average cost of $0.006 per skill. An optional XGBoost fast path reduces Layer-2/3 calls by 32% with a 1.7-point F1 decrease while preserving recall. On 52 Feishu/Lark packages, Layer 2 reclassifies 13 of 14 Layer-1 flags as safe after contextual analysis; we also deploy the system as a Feishu chat bot. Code, labels, and aggregate results are open-sourced.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。