提出新基准,评估智能体安全监控的干预时机是否及时。
StepShield: When, Not Whether to Intervene on Rogue Agents
- 定义早期干预率(EIR)衡量警报触发的及时性
- 发现规则型监控86%召回率但干预时间随机,实际无效
- 揭示现有方法无法兼顾高召回、低误报与及时干预
当前智能体安全评测只关注能否检测到危害,而非何时检测。我们提出StepShield,首个将检测时机作为核心指标的基准。在9,429条基于代码的智能体行为轨迹上,定义早期干预率(EIR):在偏离点后k步内发出警报的检测比例,剥离覆盖度影响,专注评估时机。结果揭示‘事后追查陷阱’:一个含847条规则的模式防护系统虽有86%召回率,但其EIR为0.23,与随机水平(0.24)无显著差异(p=0.66,单侧二项检验;差值在置信区间内),因超过四分之三的警报在无违规前的良性代码阶段就已触发。规则型与语义型检测器间4倍的EIR差距,在传统指标如准确率、召回率或F1中完全不可见。根本原因在于正则规则仅识别语法,无法判断意图,故无法捕捉智能体变坏的精确时刻,表明现有基于模式的监控系统不适用于实时监管。目前尚无方法能同时实现高召回、低误报与及时干预,证实了步级层面的恶意行为检测仍是未解难题。
原文摘要 · Abstract (English)
Agent safety benchmarks measure whether a monitor detects harm, not when. Yet timing is the difference between intervention and autopsy. We introduce StepShield, the first benchmark that treats detection timeliness as a first-class metric. On 9,429 incident-grounded code-agent trajectories, we define the Early Intervention Rate (EIR): the fraction of detected rogue trajectories where the alert fires within a k-step window after the divergence point, isolating timing quality from coverage. This metric exposes what we call the Forensics Trap: a pattern-based guardrail with 847 rules achieves 86% recall yet is statistically indistinguishable from random timing on EIR (0.23 vs. 0.24; p = 0.66, one-sided binomial; difference within CI), because over three-quarters of its alerts trigger on benign prefix code before any violation occurs. The 4x EIR gap between rule-based and semantic detectors is completely invisible to accuracy, recall, or F1. Our finding is structural: regex guardrails detect syntax, not intent, and therefore cannot distinguish the moment an agent turns rogue, rendering the entire deployed class of pattern-based monitors unsuited for real-time oversight. No existing method simultaneously achieves high recall, low false-positive rate, and timely intervention, establishing step-level rogue detection as genuinely unsolved.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。