提出可检测大模型泄露风险的轻量级行为探测器。
The Model's Tell: Measuring Context-Leakage Attack Signals with Behavior Gauges

- 通过尾缀探测响应概率,生成攻击风险评分。
- 在11个大模型上实现0.944~0.996的AUROC,跨语言稳定。
- 仅需不到500参数,延迟仅10.34毫秒,适合部署。
大语言模型依赖外部上下文(如系统提示或检索文档)提升生成质量,但这也带来安全风险:恶意输入可能诱使模型泄露这些内容。已有研究指出泄露信号存在于隐藏状态,但提取难度高。本文提出LeakGauge,通过添加尾缀探测响应概率,将预填充阶段的词元概率映射为攻击风险分数。相比直接利用敏感内容开头的探测方式,采用不依赖内容的泛化型尾缀(即描述泄露行为)能获得更稳健信号。在包括GLM-5.2(753B)和Kimi-K3(2.8T)在内的11个模型上,LeakGauge对未见攻击的AUROC达到0.944–0.996,且在内容换语种或从字面泄露转为语义泄露时仍保持稳定。激活干预实验表明该风险分数与模型内部泄露相关方向高度敏感。此外,该方法支持仅需<0.5K额外参数、延迟仅10.34毫秒的输入检测器部署。
原文摘要 · Abstract (English)
LLMs increasingly rely on external contexts, such as pre-defined system prompts or retrieved documents, to improve generation quality. However, processing these contexts alongside user queries creates an attack surface: adversarial inputs can induce models to disclose them. Prior probing studies suggest that leakage-related signals emerge in hidden states, yet the need to extract these states poses additional deployment challenges. In this paper, we explore whether this internal signal leaves a more accessible ``tell'' before decoding. We propose LeakGauge, which probes this response by appending a suffix that gauges leakage behavior and mapping its prefill token probabilities to an attack-risk score. While a direct gauge uses the initial tokens of confidential content, we find that a content-agnostic one that verbalizes leakage behavior yields more robust signals. Across 11 LLMs, including GLM-5.2 (753B) and Kimi-K3 (2.8T), LeakGauge reaches an AUROC range of 0.944--0.996 on unseen attacks. The signal remains stable when the content changes language or the attack shifts from verbatim to semantic disclosure. By activation-steering interventions, we further show that the risk score is sensitive to an internal leakage-related direction, relating the observable signal to the model's internal representation. In addition, LeakGauge enables an input detector with fewer than 0.5K extra parameters and added latency of 10.34 ms. Code: \href{https://github.com/yeasen-z/LeakGauge}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。