arXiv:2508.03936cs.CRcs.CL2025-08被引 6

ASTRA自动挖掘AI编程助手的真实漏洞,提升安全性和可靠性。

ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistants

  • 构建领域知识图谱,动态探索模型输入与推理过程
  • 比现有方法多发现11%-66%的安全漏洞
  • 适合安全研究人员和开发团队用于模型对齐训练

AI编程助手如GitHub Copilot正快速改变软件开发,但其安全性在高风险领域(如网络安全)仍存重大不确定性。现有红队测试工具常依赖固定基准或不现实的提示,难以发现真实世界漏洞。我们提出ASTRA,一种自动化智能体系统,旨在系统性地发现基于AI的代码生成与安全指导系统的安全缺陷。ASTRA分三阶段运行:(1) 构建结构化的特定领域知识图谱,建模复杂软件任务与已知弱点;(2) 基于知识图谱,通过自适应探测目标模型的输入空间(空间探索)与推理过程(时间探索),实现在线漏洞挖掘;(3) 生成高质量违规诱导案例,以提升模型对齐。与以往方法不同,ASTRA聚焦开发者可能提出的实际请求,并结合离线抽象建模与在线知识图谱适配,揭示边缘情况漏洞。在两个主要评估领域中,ASTRA比现有技术多发现11%-66%的问题,并生成使对齐训练效果提升17%的测试用例,证明其在构建更安全AI系统中的实用价值。

原文摘要 · Abstract (English)

AI coding assistants like GitHub Copilot are rapidly transforming software development, but their safety remains deeply uncertain-especially in high-stakes domains like cybersecurity. Current red-teaming tools often rely on fixed benchmarks or unrealistic prompts, missing many real-world vulnerabilities. We present ASTRA, an automated agent system designed to systematically uncover safety flaws in AI-driven code generation and security guidance systems. ASTRA works in three stages: (1) it builds structured domain-specific knowledge graphs that model complex software tasks and known weaknesses; (2) it performs online vulnerability exploration of each target model by adaptively probing both its input space, i.e., the spatial exploration, and its reasoning processes, i.e., the temporal exploration, guided by the knowledge graphs; and (3) it generates high-quality violation-inducing cases to improve model alignment. Unlike prior methods, ASTRA focuses on realistic inputs-requests that developers might actually ask-and uses both offline abstraction guided domain modeling and online domain knowledge graph adaptation to surface corner-case vulnerabilities. Across two major evaluation domains, ASTRA finds 11-66% more issues than existing techniques and produces test cases that lead to 17% more effective alignment training, showing its practical value for building safer AI systems.

AI安全红队测试代码生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。