用模型自身隐状态实现零成本实时安全监控,提升生成可靠性。
SingProbe Technical Report

- 直接复用推理时的隐藏状态,无需额外模型
- 仅需约200万参数,额外开销低于0.5%
- 可实时检测风险并指导安全生成,适合医疗等高敏感场景
运行时防护对大语言模型的可靠部署至关重要,但现有方法多依赖独立的外部模型,带来额外推理开销、延迟的安全信号和与日益强大的基础模型之间的能力不匹配问题。为此,我们提出SingProbe,一种轻量级内生式运行时防护机制,直接复用大语言模型推理过程中产生的隐藏状态,并与自回归解码并行运行。在统一框架下,SingProbe以极低额外开销在词元级别持续预测查询意图、响应安全性和幻觉风险,实现‘免费午餐’式的防护效果。我们进一步构建了SingStreamBench基准,用于评估流式防护在良性前缀上保持静默的同时,能否及时发现潜在不安全内容。大量实验表明,SingProbe在性能上达到甚至超越显著更大的独立防护模型和专用幻觉检测器,仅需约200万参数且额外开销小于0.5%。除被动检测外,我们还证明了SingProbe得分可预判未来生成风险并引导受限安全解码。此外,我们将该范式扩展至医疗生成领域,提出SingProbe-Med,仅在出现临床相关风险时才选择性激活风险导向的解码干预。这些结果共同表明,内部模型表示为生成过程中的监控与控制提供了有效且高效的接口。
原文摘要 · Abstract (English)
Runtime guardrails are essential for reliable large language model (LLM) deployment, yet existing approaches typically rely on independent, external models that introduce additional inference cost, delayed safety signals, and a capacity mismatch with increasingly capable base models. To address these issues, we introduce SingProbe, a lightweight intrinsic runtime guard that directly reuses hidden states produced during LLM inference and operates alongside autoregressive decoding. Within a unified framework, SingProbe continuously predicts query intent, response safety, and hallucination risk at the token level with negligible additional guardrail inference overhead, offering a "free-lunch" solution. We further introduce SingStreamBench, a benchmark designed to assess whether streaming guardrails remain inactive on benign prefixes while promptly detecting emerging unsafe content. Extensive experiments show that SingProbe achieves competitive or superior performance compared with substantially larger standalone guardrails and specialized hallucination detectors, with only $\approx$2M parameters and $<0.5\%$ extra overhead. Beyond passive detection, we also show that SingProbe scores can anticipate future generation risk and guide constrained safe decoding. We further extend this paradigm to medical generation through SingProbe-Med, which selectively activates risk-directed decoding interventions only when clinically relevant risks emerge. Together, these results demonstrate that internal model representations provide an effective and efficient interface for generation-time monitoring and control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。