arXiv:2510.19844cs.CRcs.AI2025-10被引 2

用多智能体辩论机制,更精准识别恶意提示攻击。

CourtGuard: A Local, Multiagent Prompt Injection Classifier

  • 让辩护、控诉、裁判三模型辩论提示内容。
  • 误报率低于直接判别法,但检测能力较弱。
  • 适合关注安全性的系统集成与对抗测试。

随着大语言模型(LLMs)在敏感场景中广泛应用,提示注入攻击——即通过特定提示诱导模型产生有害行为——风险日益加剧。此类攻击可能导致数据泄露、传播虚假信息或引发不当行为。为应对该威胁,本文提出 CourtGuard,一种本地运行的多智能体提示注入分类器。其核心机制为:在类法庭环境中,由‘辩护律师’模型主张提示无害,‘控诉律师’模型主张其为提示注入攻击,最终由‘法官’模型作出分类判定。实验表明,CourtGuard 的误报率低于直接判别法(Direct Detector),尽管整体检测性能较差,但更低的误报率凸显了在分类任务中兼顾对抗性与良性场景的重要性。此外,该方法推动了多智能体系统在防御提示注入攻击中的应用。Gemma-3-12b-it、Llama-3.3-8B 和 Phi-4-mini-instruct 的完整实现已开源于 https://github.com/isaacwu2000/CourtGuard。

原文摘要 · Abstract (English)

As large language models (LLMs) become integrated into various sensitive applications, prompt injection, the use of prompting to induce harmful behaviors from LLMs, poses an ever increasing risk. Prompt injection attacks can cause LLMs to leak sensitive data, spread misinformation, and exhibit harmful behaviors. To defend against these attacks, we propose CourtGuard, a locally-runnable, multiagent prompt injection classifier. In it, prompts are evaluated in a court-like multiagent LLM system, where a "defense attorney" model argues the prompt is benign, a "prosecution attorney" model argues the prompt is a prompt injection, and a "judge" model gives the final classification. CourtGuard has a lower false positive rate than the Direct Detector, an LLM as-a-judge. However, CourtGuard is generally a worse prompt injection detector. Nevertheless, this lower false positive rate highlights the importance of considering both adversarial and benign scenarios for the classification of a prompt. Additionally, the relative performance of CourtGuard in comparison to other prompt injection classifiers advances the use of multiagent systems as a defense against prompt injection attacks. The implementations of CourtGuard and the Direct Detector with full prompts for Gemma-3-12b-it, Llama-3.3-8B, and Phi-4-mini-instruct are available at https://github.com/isaacwu2000/CourtGuard.

提示攻击多智能体安全防御

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。