arXiv:2509.00088cs.CRcs.AI2025-09被引 2

自动协同演化框架提升对抗提示注入攻击的防御能力。

AEGIS : Automated Co-Evolutionary Framework for Guarding Prompt Injections Schema

  • 攻防提示通过文本梯度优化迭代进化,实现自适应对抗。
  • 攻击成功率达1.0,检测真阳性率提升至0.84,优于基线。
  • 适用于多种大模型,适合安全研究与工业部署场景。

提示注入攻击对大型语言模型在真实场景中的安全部署构成重大挑战。尽管基于提示的检测方法具有轻量且可解释的优势,但其效果受限于人工提示工程。为此,我们提出AEGIS——一种用于防御提示注入模式的自动化协同演化框架。该框架利用类梯度的语言提示优化技术,使攻击与防御提示在LLM引导的评估反馈循环中相互迭代优化。我们在真实作业评分数据集上评估了该系统,结果表明,本方法在攻击成功率和检测性能上均显著优于现有基线:攻击成功率达1.0,较基线提升0.26;检测真阳性率(TPR)提升0.23,达到0.84,真阴性率(TNR)保持在0.89。消融实验验证了协同演化、梯度缓冲及多目标优化的重要性。结果还表明该框架在不同大模型上均有效,凸显对抗训练在防御提示注入方面的可扩展性与有效性。

原文摘要 · Abstract (English)

Prompt injection attacks pose a significant challenge to the safe deployment of Large Language Models (LLMs) in real-world applications. While prompt-based detection offers a lightweight and interpretable defense strategy, its effectiveness has been hindered by the need for manual prompt engineering. To address this issue, we propose AEGIS , an Automated co-Evolutionary framework for Guarding prompt Injections Schema. Both attack and defense prompts are iteratively optimized against each other using a gradient-like natural language prompt optimization technique. This framework enables both attackers and defenders to autonomously evolve via a Textual Gradient Optimization (TGO) module, leveraging feedback from an LLM-guided evaluation loop. We evaluate our system on a real-world assignment grading dataset of prompt injection attacks and demonstrate that our method consistently outperforms existing baselines, achieving superior robustness in both attack success and detection. Specifically, the attack success rate (ASR) reaches 1.0, representing an improvement of 0.26 over the baseline. For detection, the true positive rate (TPR) improves by 0.23 compared to the previous best work, reaching 0.84, and the true negative rate (TNR) remains comparable at 0.89. Ablation studies confirm the importance of co-evolution, gradient buffering, and multi-objective optimization. We also confirm that this framework is effective in different LLMs. Our results highlight the promise of adversarial training as a scalable and effective approach for guarding prompt injections.

提示攻击对抗训练安全防护大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。