arXiv:2509.11398cs.LGcs.AI2025-09被引 4

将AI红队视为网络安全红队的演化,提升对智能系统漏洞的评估能力。

From Firewalls to Frontiers: AI Red-Teaming is a Domain-Specific Evolution of Cyber Red-Teaming

  • 把AI红队看作网络安全红队的延伸,统一评估框架。
  • 揭示AI系统存在不可修复漏洞,需重新规划披露与应对策略。
  • 适合安全团队、AI开发者及政策制定者参考,推动协同防御。

红队通过模拟攻击帮助防御方在真实操作环境中发现有效防御策略。随着企业系统越来越多采用人工智能,红队必须演进以应对AI带来的独特脆弱性和风险。我们认为,若将AI红队视为网络安全红队的领域特化发展,则能更有效地评估含AI组件的系统。具体而言,现有网络安全红队采用此视角后,可识别出AI带来的新风险、新的可利用故障模式,并意识到许多缺陷无法修补,从而重新优先安排披露与缓解策略。同样,采用网络安全框架能使现有AI红队借助成熟结构模拟真实对手,建立正式行为规则以促进相互问责,并形成可复用、可扩展的工具体系。两者融合将构建强健的安全生态,更好应对快速变化的威胁环境。

原文摘要 · Abstract (English)

A red team simulates adversary attacks to help defenders find effective strategies to defend their systems in a real-world operational setting. As more enterprise systems adopt AI, red-teaming will need to evolve to address the unique vulnerabilities and risks posed by AI systems. We take the position that AI systems can be more effectively red-teamed if AI red-teaming is recognized as a domain-specific evolution of cyber red-teaming. Specifically, we argue that existing Cyber Red Teams who adopt this framing will be able to better evaluate systems with AI components by recognizing that AI poses new risks, has new failure modes to exploit, and often contains unpatchable bugs that re-prioritize disclosure and mitigation strategies. Similarly, adopting a cybersecurity framing will allow existing AI Red Teams to leverage a well-tested structure to emulate realistic adversaries, promote mutual accountability with formal rules of engagement, and provide a pattern to mature the tooling necessary for repeatable, scalable engagements. In these ways, the merging of AI and Cyber Red Teams will create a robust security ecosystem and best position the community to adapt to the rapidly changing threat landscape.

红队演练AI安全网络安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。