用进化方法自动发现大模型智能体的安全漏洞,比传统测试更有效。
NAAMSE: Framework for Evolutionary Security Evaluation of Agents
- 设计一个自进化框架,通过基因突变提示不断优化攻击策略。
- 在多个顶尖大模型上发现一击式测试遗漏的高危漏洞,成功率提升显著。
- 适合安全研究人员和模型开发者,用于评估智能体应对复杂攻击的能力。
AI智能体在生产中日益普及,但其安全评估仍受限于人工红队或静态基准,无法模拟适应性强、多轮交互的对抗者。本文提出NAAMSE,一种将智能体安全评估重构为反馈驱动优化问题的进化框架。系统由单一自主代理执行基因化提示变异、分层语料探索与非对称行为评分的生命周期。利用模型响应作为适应度信号,框架可迭代强化有效攻击策略,同时确保“良性使用正确性”,避免因一味拒绝导致的安全退化。在一系列前沿大语言模型上的实验表明,进化变异能系统性放大一击式方法遗漏的漏洞,受控消融分析显示探索与定向变异的协同作用揭示了高严重性失效模式。该自适应方法为应对演化威胁提供了更真实、可扩展的鲁棒性评估。代码已开源,地址:https://github.com/HASHIRU-AI/NAAMSE。
原文摘要 · Abstract (English)
AI agents are increasingly deployed in production, yet their security evaluations remain bottlenecked by manual red-teaming or static benchmarks that fail to model adaptive, multi-turn adversaries. We propose NAAMSE, an evolutionary framework that reframes agent security evaluation as a feedback-driven optimization problem. Our system employs a single autonomous agent that orchestrates a lifecycle of genetic prompt mutation, hierarchical corpus exploration, and asymmetric behavioral scoring. By using model responses as a fitness signal, the framework iteratively compounds effective attack strategies while simultaneously ensuring "benign-use correctness", preventing the degenerate security of blanket refusal. Our experiments across a diverse suite of state-of-the-art large language models demonstrate that evolutionary mutation systematically amplifies vulnerabilities missed by one-shot methods, with controlled ablations revealing that the synergy between exploration and targeted mutation uncovers high-severity failure modes. We show that this adaptive approach provides a more realistic and scalable assessment of agent robustness in the face of evolving threats. The code for NAAMSE is open source and available at https://github.com/HASHIRU-AI/NAAMSE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。