评测主流提示注入检测系统，发现漏洞并提出改进方案。
Enhancing Security in LLM Applications: A Performance Evaluation of Early Detection Systems
- 对比 LLM Guard、Vigil、Rebuff 三种开源检测工具的实现机制。
- 发现 Vigil 与 Rebuff 的蜜罐词检查无效，且 Rebuff 存在绕过漏洞。
- 推荐 Vigil 用于低误报场景，Rebuff 适合通用部署。
提示注入攻击威胁基于大语言模型（LLM）的新应用安全。本文评估了早期提示注入检测系统的能力，重点关注多种开源解决方案中实现的检测技术。这些方案旨在识别包括提示泄露在内的攻击类型——即攻击者操纵模型输出其系统指令，破坏系统机密性。我们分析了不同提示泄露检测技术，并对 LLM Guard、Vigil、Rebuff 进行了对比研究。结果表明，Vigil 和 Rebuff 中的蜜罐词检查对提示泄露攻击无效；同时发现 Rebuff 的二级模型检测存在可被规避的弱点。在性能最优条件下，我们建议：若需极低误报率，应选用 Vigil；若满足一般需求，则推荐 Rebuff。
原文摘要 · Abstract (English)
Prompt injection threatens novel applications that emerge from adapting LLMs for various user tasks. The newly developed LLM-based software applications become more ubiquitous and diverse. However, the threat of prompt injection attacks undermines the security of these systems as the mitigation and defenses against them, proposed so far, are insufficient. We investigated the capabilities of early prompt injection detection systems, focusing specifically on the detection performance of techniques implemented in various open-source solutions. These solutions are supposed to detect certain types of prompt injection attacks, including the prompt leak. In prompt leakage attacks, an attacker maliciously manipulates the LLM into outputting its system instructions, violating the system's confidentiality. Our study presents analyzes of distinct prompt leakage detection techniques, and a comparative analysis of several detection solutions, which implement those techniques. We identify the strengths and weaknesses of these techniques and elaborate on their optimal configuration and usage in high-stake deployments. In one of the first studies on existing prompt leak detection solutions, we compared the performances of LLM Guard, Vigil, and Rebuff. We concluded that the implementations of canary word checks in Vigil and Rebuff were not effective at detecting prompt leak attacks, and we proposed improvements for them. We also found an evasion weakness in Rebuff's secondary model-based technique and proposed a mitigation. Then, the result of the comparison of LLM Guard, Vigil, and Rebuff at their peak performance revealed that Vigil is optimal for cases when minimal false positive rate is required, and Rebuff is the most optimal for average needs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。