首次系统评估大模型在多步工具调用中的安全防护能力，发现结构理解比语义对齐更重要。
TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories
- 构建首个针对中间执行轨迹的安全评测基准TraceSafe-Bench，覆盖12类风险。
- 模型结构解析能力与安全检测相关性高达0.79，远超传统越狱鲁棒性测试。
- 通用大模型比专用安全模型更有效，长轨迹下过拒现象导致性能下降。
随着大语言模型从静态聊天机器人演变为自主代理，主要安全威胁已从最终输出转移至中间执行轨迹。尽管对自然语言响应的安全防护已有充分评估，但对多步工具使用轨迹中的安全性仍缺乏系统研究。为此，我们提出**TraceSafe-Bench**，首个专为评估中段轨迹安全设计的综合性基准，涵盖12类风险，包括安全威胁（如提示注入、隐私泄露）和操作失败（如幻觉、接口不一致），包含超过1,000个独特执行实例。我们评估了13个以LLM为防护机制的模型和7个专用防护策略，得出三个关键发现：1）结构性瓶颈：防护效果更多取决于结构化数据处理能力（如JSON解析），而非语义安全对齐；性能与结构化到文本的基准相关性达$ρ=0.79$（$p<0.01$），而与标准越狱鲁棒性无显著关联。2）规模并非决定因素：轨迹检测准确率不随模型规模单调提升，通用大模型始终优于专用安全防护。3）有限时间稳定性：在原生上下文范围内，检测准确率随轨迹长度增加而提升，因模型能动态处理执行行为；但在极端长上下文场景中，因过度拒绝导致性能下降。
原文摘要 · Abstract (English)
As large language models (LLMs) evolve from static chatbots into autonomous agents, the primary vulnerability surface shifts from final outputs to intermediate execution traces. While safety guardrails are well-benchmarked for natural language responses, their efficacy remains largely unexplored within multi-step tool-use trajectories. To address this gap, we introduce **TraceSafe-Bench**, the first comprehensive benchmark specifically designed to assess mid-trajectory safety. It encompasses 12 risk categories, ranging from security threats (e.g., prompt injection, privacy leaks) to operational failures (e.g., hallucinations, interface inconsistencies), featuring over 1,000 unique execution instances. Our evaluation of 13 LLM-as-a-guard models and 7 specialized guardrails yields three critical findings: 1) *Structural Bottleneck*: Guardrail efficacy is driven more by structural data competence (e.g., JSON parsing) than semantic safety alignment. Performance correlates strongly with structured-to-text benchmarks ($ρ=0.79$, $p<0.01$) while we detect no association with standard jailbreak robustness. 2) *Scale Is Not the Deciding Factor*: Trajectory detection accuracy does not scale monotonically with model size, and general-purpose LLMs consistently outperform specialized safety guardrails. 3) *Bounded Temporal Stability*: Detection accuracy improves with trajectory length within native context bounds as models process dynamic execution behavior, but degrades in extreme long-context regimes due to over-refusal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。