通过引入人物角色提升自动化红队测试效果,发现角色多样性显著增强攻击成功率。
PersonaTeaming: Exploring How Introducing Personas Can Improve Automated AI Red-Teaming
- 在对抗性提示生成中引入专家与普通用户等角色设定
- 攻击成功率最高提升144.1%,同时保持提示多样性
- 适合关注AI安全评估与自动化测试的科研人员
近期人工智能治理与安全研究呼吁更有效的红队测试方法以揭示模型潜在风险。研究表明,红队人员的身份背景会影响其策略选择与风险发现范围。尽管自动化红队测试可实现大规模行为探索,但现有方法忽视了身份因素的影响。为此,我们提出PersonaTeaming,一种在对抗性提示生成中引入人物角色的新方法,以拓展攻击策略的多样性。具体包括:基于“红队专家”或“普通用户”角色构建提示变异机制;设计动态角色生成算法,根据初始提示自适应生成不同角色类型;开发新度量指标“突变距离”,补充现有提示多样性评估。实验表明,相比当前最先进的RainbowPlus方法,该方法在攻击成功率上提升最高达144.1%,且维持了提示多样性。我们还分析了不同角色类型与变异方法的优劣,为未来自动化与人工红队协同提供了新思路。
原文摘要 · Abstract (English)
Recent developments in AI governance and safety research have called for red-teaming methods that can effectively surface potential risks posed by AI models. Many of these calls have emphasized how the identities and backgrounds of red-teamers can shape their red-teaming strategies, and thus the kinds of risks they are likely to uncover. While automated red-teaming approaches promise to complement human red-teaming by enabling larger-scale exploration of model behavior, current approaches do not consider the role of identity. As an initial step towards incorporating people's background and identities in automated red-teaming, we develop and evaluate a novel method, PersonaTeaming, that introduces personas in the adversarial prompt generation process to explore a wider spectrum of adversarial strategies. In particular, we first introduce a methodology for mutating prompts based on either "red-teaming expert" personas or "regular AI user" personas. We then develop a dynamic persona-generating algorithm that automatically generates various persona types adaptive to different seed prompts. In addition, we develop a set of new metrics to explicitly measure the "mutation distance" to complement existing diversity measurements of adversarial prompts. Our experiments show promising improvements (up to 144.1%) in the attack success rates of adversarial prompts through persona mutation, while maintaining prompt diversity, compared to RainbowPlus, a state-of-the-art automated red-teaming method. We discuss the strengths and limitations of different persona types and mutation methods, shedding light on future opportunities to explore complementarities between automated and human red-teaming approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。