区分大模型工具代理的语义攻击、审计证据与实际危害,精准定位安全漏洞。
SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents
- 设计六类攻击任务,分三阶段评估模型行为:语义接受、审计证据、沙箱执行
- 44.2%模型在无防护下发生语义失败,347个任务中291个通过语义检查却仍造成实际危害
- 揭示提示词策略效果依赖模型和协议,为安全评估提供可复现框架
使用工具的语言模型代理引入的安全问题已超越不安全文本:可能泄露受保护对象、写入持久内存、发送消息、修改数据库或触发有害代码与工具效应。现有评估常将各阶段合并为单一成功率,难以判断模型是仅同意攻击者还是真正产生可观测危害。我们提出SafeClawBench,一个分阶段的工具型代理安全评测基准,包含600个受控对抗任务,覆盖六类攻击:直接与间接提示注入、工具返回注入、内存污染、内存提取及模糊驱动的不安全推理。该基准报告三个独立指标:语义攻击接受度、审计可见危害证据、沙箱中观测到的工具/状态危害。在四种提示级策略下评估五个代理端点,发现三类指标捕捉不同失效模式。无额外提示防护时,语义失败率在9.0%至44.2%间波动。审计危害证据范围更窄,且在12,000行匹配分析中,347例沙箱危害中有291例发生在通过语义核心检测的样本中。提示策略会改变指标结果,但其影响取决于模型与协议。SafeClawBench提供可复现的框架,用于比较代理模型与提示策略,避免混淆文本合规性、证据支持的危害和可执行的状态变化。开源数据集见https://huggingface.co/datasets/sairights/safeclawbench。
原文摘要 · Abstract (English)
Tool-using language-model agents introduce security failures that go beyond unsafe text: they can disclose protected objects, write persistent memory, send messages, modify databases, or trigger harmful code and tool effects. Existing evaluations often collapse these stages into a single attack success rate, making it difficult to tell whether a model merely agreed with an attacker or actually produced observable harm. We introduce SafeClawBench, a staged benchmark for tool-using agent security with 600 controlled adversarial tasks across six attack families: direct and indirect prompt injection, tool-return injection, memory poisoning, memory extraction, and ambiguity-driven unsafe inference. SafeClawBench reports three separate endpoints: semantic attack acceptance, audit-visible harm evidence, and sandbox-observed tool/state harm. Evaluating five agent endpoints under four prompt-level policies, we find that these endpoints capture different failure modes. Without additional prompt protection, semantic failure rates vary widely across models, from 9.0% to 44.2%. Audited harm evidence is narrower than semantic failure, and under a separate executable protocol some matched task identities produce sandbox harm despite passing the Semantic Core call: in a 12,000-row matched analysis, 291 of 347 observed sandbox harms occur in rows that pass the semantic check. Prompt policies change endpoint outcomes, but their effects depend on both model and protocol. SafeClawBench provides a reproducible framework for comparing agent models and prompt-policy conditions without conflating textual compliance, evidence-supported harm, and executable state changes. The open-source dataset is available at https://huggingface.co/datasets/sairights/safeclawbench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。