用智能搜索发现隐藏在大量智能体轨迹中的安全漏洞
Detecting Safety Violations Across Many Agent Traces
- 通过聚类与智能体搜索结合，自动定位潜在违规区域
- 在多个测试中检出近4倍于以往的奖励操控案例
- 适合审计复杂系统中的隐蔽违规行为，如作弊或对抗攻击
为识别安全违规，审计人员常需分析大量智能体轨迹。此类搜索困难源于故障稀少、复杂且可能被刻意隐藏，仅在多条轨迹联合分析时才可暴露。此类问题出现在滥用行为、隐秘破坏、奖励黑客和提示注入等多种场景。现有方法存在局限：单轨迹判别器会漏掉跨轨迹违规，盲目代理审计无法扩展至大规模轨迹集，固定监控器对未预见行为敏感。本文提出Meerkat，结合聚类与智能搜索，以自然语言描述的规则检测违规。通过结构化搜索与对高潜力区域的自适应调查，无需种子场景、固定流程或全量枚举即可发现稀疏故障。在滥用、对齐偏差及任务博弈等场景中，Meerkat显著优于基线监控器；在顶级代理基准上发现广泛开发者作弊，并在CyBench数据集上比此前审计多发现近4倍的奖励黑客实例。
原文摘要 · Abstract (English)
To identify safety violations, auditors often search over large sets of agent traces. This search is difficult because failures are often rare, complex, and sometimes even adversarially hidden and only detectable when multiple traces are analyzed together. These challenges arise in diverse settings such as misuse campaigns, covert sabotage, reward hacking, and prompt injection. Existing approaches struggle here for several reasons. Per-trace judges miss failures that only become visible across traces, naive agentic auditing does not scale to large trace collections, and fixed monitors are brittle to unanticipated behaviors. We introduce Meerkat, which combines clustering with agentic search to uncover violations specified in natural language. Through structured search and adaptive investigation of promising regions, Meerkat finds sparse failures without relying on seed scenarios, fixed workflows, or exhaustive enumeration. Across misuse, misalignment, and task gaming settings, Meerkat significantly improves detection of safety violations over baseline monitors, discovers widespread developer cheating on a top agent benchmark, and finds nearly 4x more examples of reward hacking on CyBench than previous audits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。