提出防御RAG系统提示注入攻击的多层框架，有效将攻击成功率降至8.7%。
Securing AI Agents Against Prompt Injection Attacks
- 构建包含847个用例的综合测试集，覆盖五类攻击方式。
- 多层防护使攻击成功率达8.7%，任务性能保持94.3%。
- 适合关注AI代理安全与RAG系统防护的研究者使用。
检索增强生成（RAG）系统广泛用于提升大语言模型能力，但易受提示注入攻击影响。本文提出一个全面的评估基准，用于检测RAG驱动的AI代理中的提示注入风险，并设计了一套多层防御框架。该基准包含847个对抗性测试用例，覆盖五类攻击：直接注入、上下文操纵、指令覆盖、数据外泄和跨上下文污染。我们在七种先进语言模型上评估了三种防御机制：基于嵌入的异常检测内容过滤、分层系统提示防护以及多阶段响应验证。联合使用时，攻击成功率从73.2%降至8.7%，同时保持94.3%的基线任务性能。相关数据集与防御实现已公开，以支持未来研究。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) systems have become widely used for enhancing large language model capabilities, but they introduce significant security vulnerabilities through prompt injection attacks. We present a comprehensive benchmark for evaluating prompt injection risks in RAG-enabled AI agents and propose a multi-layered defense framework. Our benchmark includes 847 adversarial test cases across five attack categories: direct injection, context manipulation, instruction override, data exfiltration, and cross-context contamination. We evaluate three defense mechanisms: content filtering with embedding-based anomaly detection, hierarchical system prompt guardrails, and multi-stage response verification, across seven state-of-the-art language models. Our combined framework reduces successful attack rates from 73.2% to 8.7% while maintaining 94.3% of baseline task performance. We release our benchmark dataset and defense implementation to support future research in AI agent security.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。