arXiv:2508.00602cs.CRcs.AI2025-08

提出防御大模型提示注入与数据泄露的新框架,兼顾静态分析与动态拦截。

LeakSealer: A Semisupervised Defense for LLMs Against Prompt Injection and Leakage Attacks

  • 基于历史交互生成主题使用图谱,识别恶意攻击模式。
  • 动态检测中隐私信息泄露的AUPRC达0.97,优于基线模型。
  • 适合安全团队用于监控和防护生产环境的大模型系统。

大型语言模型(LLMs)的泛化能力使其在各类应用中广泛部署,但也带来了越狱和数据泄露等安全威胁。检索增强生成(RAG)虽提升了上下文感知能力,却意外引入了敏感信息泄露漏洞。本文提出双重贡献:首先,通过分析历史交互数据生成按主题分类的使用图谱,实现对越狱攻击演变的溯源分析;其次,提出模型无关的LeakSealer框架,结合静态分析与人机协同的动态防御机制,可识别话题群组并检测异常模式。我们在两种场景下评估该方法:(1)越狱攻击,使用公开的ToxicChat数据集;(2)个人身份信息(PII)泄露,基于标注的交互数据集。静态设置下,LeakSealer在识别提示注入方面达到最高精确率与召回率;动态设置中,PII泄露检测的AUPRC为0.97,显著优于Llama Guard等基线模型。

原文摘要 · Abstract (English)

The generalization capabilities of Large Language Models (LLMs) have led to their widespread deployment across various applications. However, this increased adoption has introduced several security threats, notably in the forms of jailbreaking and data leakage attacks. Additionally, Retrieval Augmented Generation (RAG), while enhancing context-awareness in LLM responses, has inadvertently introduced vulnerabilities that can result in the leakage of sensitive information. Our contributions are twofold. First, we introduce a methodology to analyze historical interaction data from an LLM system, enabling the generation of usage maps categorized by topics (including adversarial interactions). This approach further provides forensic insights for tracking the evolution of jailbreaking attack patterns. Second, we propose LeakSealer, a model-agnostic framework that combines static analysis for forensic insights with dynamic defenses in a Human-In-The-Loop (HITL) pipeline. This technique identifies topic groups and detects anomalous patterns, allowing for proactive defense mechanisms. We empirically evaluate LeakSealer under two scenarios: (1) jailbreak attempts, employing a public benchmark dataset, and (2) PII leakage, supported by a curated dataset of labeled LLM interactions. In the static setting, LeakSealer achieves the highest precision and recall on the ToxicChat dataset when identifying prompt injection. In the dynamic setting, PII leakage detection achieves an AUPRC of $0.97$, significantly outperforming baselines such as Llama Guard.

大模型安全提示注入数据泄露

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。