利用大模型内部特征检测提示注入攻击，效果好且效率高。
PIShield: Detecting Prompt Injection Attacks via Intrinsic LLM Features
- 通过分析模型残差流特征识别恶意指令注入
- 在多类测试中误报率和漏报率均显著低于现有方法
- 无需微调或生成响应，适合实际应用部署
集成大语言模型的应用易受提示注入攻击影响，攻击者通过污染输入注入恶意指令，使模型偏离用户原意。现有检测方法普遍存在性能不足或计算开销过高的问题。本文提出PIShield，基于指令微调大模型内部对注入指令具有可区分信号的观察，利用残差流表示与简单线性分类器实现检测，无需昂贵的模型微调或响应生成。我们在多种短上下文与长上下文基准上进行了全面评估，结果表明PIShield始终维持低误报率与误报率，显著优于现有基线。这些发现表明，指令微调大模型的内部表征为现实应用中的提示注入检测提供了强大而实用的基础。
原文摘要 · Abstract (English)
LLM-integrated applications are vulnerable to prompt injection attacks, where an attacker contaminates the input to inject malicious instructions, causing the LLM to follow the attacker's intent instead of the original user's. Existing prompt injection detection methods often have sub-optimal performance and/or high computational overhead. In this work, we propose PIShield, an effective and efficient detection method based on the observation that instruction-tuned LLMs internally encode distinguishable signals for prompts containing injected instructions. PIShield leverages residual-stream representations and a simple linear classifier to detect prompt injection, without expensive model fine-tuning or response generation. We conduct extensive evaluations on a diverse set of short- and long-context benchmarks. The results show that PIShield consistently achieves low false positive and false negative rates, significantly outperforming existing baselines. These findings demonstrate that internal representations of instruction-tuned LLMs provide a powerful and practical foundation for prompt injection detection in real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。