用动态变化的提示结构防御大模型提示注入攻击。
To Protect the LLM Agent Against the Prompt Injection Attack with Polymorphic Prompt
- 通过动态变换系统提示结构，让攻击者无法预测构造方式。
- 在多项攻击测试中实现近零开销的高防御效果。
- 适合需要轻量级安全防护的客服与代码辅助场景。
大型语言模型代理广泛应用于客户服务、内容生成和代码辅助等场景，但易受提示注入攻击影响，即恶意输入可操控模型行为。传统防御方法如输入清洗、守护模型和规则限制存在操作繁琐或效果不佳的问题。本文提出一种新型轻量级防御机制——多态提示组装（Polymorphic Prompt Assembling, PPA），基于提示注入需猜测并破解系统提示结构的特性，通过动态改变系统提示结构，使攻击者难以预判，从而提升安全性且不增加性能负担。实验评估显示，PPA在对抗现有攻击时表现优异，并优于多种现有防御方法。
原文摘要 · Abstract (English)
LLM agents are widely used as agents for customer support, content generation, and code assistance. However, they are vulnerable to prompt injection attacks, where adversarial inputs manipulate the model's behavior. Traditional defenses like input sanitization, guard models, and guardrails are either cumbersome or ineffective. In this paper, we propose a novel, lightweight defense mechanism called Polymorphic Prompt Assembling (PPA), which protects against prompt injection with near-zero overhead. The approach is based on the insight that prompt injection requires guessing and breaking the structure of the system prompt. By dynamically varying the structure of system prompts, PPA prevents attackers from predicting the prompt structure, thereby enhancing security without compromising performance. We conducted experiments to evaluate the effectiveness of PPA against existing attacks and compared it with other defense methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。