通过上下文白名单机制,让智能体大模型只响应安全指令,防住攻击又不误伤正常业务。
LLMZ+: Contextual Prompt Whitelist Principles for Agentic LLMs
- 用上下文感知的提示词白名单,限定智能体只能接收合规指令。
- 实验中对常见攻击提示的防御达到零误报、零漏报。
- 适合需高安全性的企业级智能体应用,如金融、医疗场景。
与传统模型相比,智能体型AI因具备访问数据源和API工具的特权,成为潜在攻击目标,其非确定性行为(仅设定目标,路径由大模型自主选择)带来显著安全风险。现有防御多依赖恶意意图检测,难以应对提示注入等越狱攻击。本文提出新型方案LLMZ+,摒弃检测思路,转而实施提示词白名单机制:仅允许与上下文匹配且安全的交互进入智能体。该方法确保所有外部用户与大模型的通信均符合预设使用场景与操作边界。实证表明,LLMZ+在抵御主流越狱攻击时表现优异,合法业务通信不受干扰,授权流量畅通无阻。评估指标显示,假阳性与假阴性率均可降至0。
原文摘要 · Abstract (English)
Compared to traditional models, agentic AI represents a highly valuable target for potential attackers as they possess privileged access to data sources and API tools, which are traditionally not incorporated into classical agents. Unlike a typical software application residing in a Demilitarized Zone (DMZ), agentic LLMs consciously rely on nondeterministic behavior of the AI (only defining a final goal, leaving the path selection to LLM). This characteristic introduces substantial security risk to both operational security and information security. Most common existing defense mechanism rely on detection of malicious intent and preventing it from reaching the LLM agent, thus protecting against jailbreak attacks such as prompt injection. In this paper, we present an alternative approach, LLMZ+, which moves beyond traditional detection-based approaches by implementing prompt whitelisting. Through this method, only contextually appropriate and safe messages are permitted to interact with the agentic LLM. By leveraging the specificity of context, LLMZ+ guarantees that all exchanges between external users and the LLM conform to predefined use cases and operational boundaries. Our approach streamlines the security framework, enhances its long-term resilience, and reduces the resources required for sustaining LLM information security. Our empirical evaluation demonstrates that LLMZ+ provides strong resilience against the most common jailbreak prompts. At the same time, legitimate business communications are not disrupted, and authorized traffic flows seamlessly between users and the agentic LLM. We measure the effectiveness of approach using false positive and false negative rates, both of which can be reduced to 0 in our experimental setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。