提出可抵抗提示注入攻击的AI代理设计模式
Design Patterns for Securing LLM Agents against Prompt Injections
- 设计六种可验证安全的防御模式，应对自然语言输入风险
- 在真实案例中验证模式有效性，平衡安全性与功能可用性
- 适合开发需处理敏感数据或调用工具的AI系统人员
随着大型语言模型（LLMs）驱动的AI代理在各类任务中愈发复杂和通用，其安全性已成为关键挑战。其中最紧迫的威胁是提示注入攻击，这类攻击利用代理对自然语言输入的脆弱性——尤其当代理具备工具访问权限或处理敏感信息时尤为危险。本文提出一套有原则的设计模式，用于构建对提示注入具备可证明抵抗力的AI代理。我们系统分析了这些模式，讨论其在效用与安全之间的权衡，并通过一系列案例研究展示其在现实场景中的适用性。
原文摘要 · Abstract (English)
As AI agents powered by Large Language Models (LLMs) become increasingly versatile and capable of addressing a broad spectrum of tasks, ensuring their security has become a critical challenge. Among the most pressing threats are prompt injection attacks, which exploit the agent's resilience on natural language inputs -- an especially dangerous threat when agents are granted tool access or handle sensitive information. In this work, we propose a set of principled design patterns for building AI agents with provable resistance to prompt injection. We systematically analyze these patterns, discuss their trade-offs in terms of utility and security, and illustrate their real-world applicability through a series of case studies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。