用AI代理将红队测试从周级缩短到小时级，自动构造攻击流程。
Redefining AI Red Teaming in the Agentic Era: From Weeks to Hours
- 通过自然语言指令让AI代理自动选择攻击、组合变换、执行并报告。
- 在多智能体系统上实现85%攻击成功率，严重性达1.0。
- 适合安全研究人员快速测试大模型漏洞，无需编写代码。
AI系统正进入医疗、金融、国防等关键领域，但仍易受对抗攻击影响。当前红队测试依赖人工手动构建攻击流程，需数周时间组装攻击、变换和评分器，若效果不佳则需重做。我们提出基于开源Dreadnode SDK的AI红队代理，可基于45+种对抗攻击、450+种变换和130+种评分器自动构建流程。用户只需描述目标，即可探测多智能体、多语言、多模态系统，聚焦于‘探什么’而非‘怎么探’。贡献包括：1. 智能接口：通过终端界面以自然语言下达任务，代理完成攻击选择、流程组合、执行与报告，时间从周级压缩至小时级；2. 统一框架：统一适用于传统机器学习模型（对抗样本）与生成式AI系统（越狱攻击），无需分库操作；3. Llama Scout案例研究：对Meta Llama Scout进行红队测试，实现85%攻击成功率，严重性最高达1.0，全程零人工代码开发。
原文摘要 · Abstract (English)
AI systems are entering critical domains like healthcare, finance, and defense, yet remain vulnerable to adversarial attacks. While AI red teaming is a primary defense, current approaches force operators into manual, library-specific workflows. Operators spend weeks hand-crafting workflows - assembling attacks, transforms, and scorers. When results fall short, workflows must be rebuilt. As a result, operators spend more time constructing workflows than probing targets for security and safety vulnerabilities. We introduce an AI red teaming agent built on the open-source Dreadnode SDK. The agent creates workflows grounded in 45+ adversarial attacks, 450+ transforms, and 130+ scorers. Operators can probe multi-agent systems, multilingual, and multimodal targets, focusing on what to probe rather than how to implement it. We make three contributions: 1. Agentic interface. Operators describe goals in natural language via the Dreadnode TUI (Terminal User Interface). The agent handles attack selection, transform composition, execution, and reporting, letting operators focus on red teaming. Weeks compress to hours. 2. Unified framework. A single framework for probing traditional ML models (adversarial examples) and generative AI systems (jailbreaks), removing the need for separate libraries. 3. Llama Scout case study. We red team Meta Llama Scout and achieve an 85% attack success rate with severity up to 1.0, using zero human-developed code
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。