通过环境演化实现可扩展的智能体红队测试,发现复杂任务中85%的安全漏洞
OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution

- 设计动态环境演化框架,用反馈驱动状态变更来暴露安全风险
- 在75种模型配置下达成85%攻击成功率,复杂任务中优势超17%
- 揭示运行时实现比模型能力更能影响安全表现,适合安全研究者使用
AI智能体在持久环境中运行,早期状态变化可能长期影响决策。与传统语言模型交互不同,智能体行为依赖于被反复修改和重用的共享状态,而现有安全基准多聚焦短时静态任务,难以捕捉累积风险。为此,我们提出OpenART——一个通过环境演化实现可扩展智能体红队测试的开放框架。OpenART提供超过10,000个经验证的状态化场景,覆盖50个领域,基于超过50万种工具和技能。这些任务平均需97次工具调用,支持对75种不同智能体模型配置的统一评估。为系统探索演化攻击面,我们提出进化马尔可夫超图攻击(EMHA),一种无需参数更新的黑盒策略,通过协调授权状态转移实现反馈驱动的环境演化。任务目标保持不变,仅环境状态变化。在所有配置中,EMHA达到85.0%的综合攻击成功率(ASR)。其相比仅指令演化的优势从简单环境的约2%提升至最复杂环境的超17%,表明环境演化在高复杂度任务中更易暴露安全缺陷。此外分析显示,智能体的具体运行时实现解释了超出模型能力之外的显著安全差异。这些结果确立OpenART作为研究复杂动态环境中智能体安全性的可扩展基础。
原文摘要 · Abstract (English)
AI agents operate in persistent environments where early state changes can influence decisions far into the future. Unlike conventional language-model interactions, agent behavior is mediated through a shared state that is repeatedly modified and reused across long-horizon workflows. Current safety benchmarks often fail to capture these cumulative risks because they focus on short, static tasks. To address these limitations, we introduce OpenART, an open-ended arena for scalable agent red teaming through environment evolution. OpenART provides over 10,000 validated stateful scenarios across 50 domains, drawing from a pool of more than 500,000 tools and skills. These tasks require a median of 97 tool calls and enable unified evaluation across 75 different agent-model configurations. To systematically explore these evolving attack surfaces, we propose the Evolutionary Markov Hypergraph Attack (EMHA). EMHA is a black-box policy that performs feedback-driven environment evolution by coordinating authorized state transitions without requiring parameter updates. Throughout the evaluation, task objectives remain fixed while only the environment state changes. Across all configurations, EMHA achieves a pooled Attack Success Rate (ASR) of 85.0%. Its advantage over instruction-only evolution increases from approximately 2% on simple environments to over 17% on the most complex ones, demonstrating that environment evolution increasingly exposes safety failures as task complexity grows. Furthermore, our analysis shows that the specific runtime implementation of an agent explains a significant portion of safety variation beyond the underlying model's capabilities. These results establish OpenART as a scalable foundation for studying agent safety in complex, evolving environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。