用模型内部状态实时检测安全风险,生成中即可拦截不当内容。
Stop Early, Spend Less: Hidden-State Probes as a Practical Recipe for Streaming Moderation of LLM Outputs

- 在模型中间层直接分析隐藏状态,生成逐标记安全评分
- 单层探针即可复现强防护模型90%以上决策,延迟低于1毫秒
- 适合对响应速度敏感的实时对话系统部署
将大语言模型应用于用户交互系统时,高效输出安全过滤至关重要。现有方法通常在生成后使用独立审核模型,导致推理成本翻倍且仅能在生成结束后发现违规。本文发现,安全判断信号已存在于模型隐藏状态中。基于此,我们训练轻量级逐标记探针,直接作用于生成器内部激活值,生成每标记的安全得分,可用于离线评估或在线干预。探针复用生成过程中的激活值,无需额外前向传播,实现解码循环内亚毫秒级每标记安全检查。单个中间层探针即可恢复强防护模型约90%的决策能力,作为低延迟替代方案,而非追求高精度。在流式场景下,可提前终止或修改不安全输出,以连续令牌级监控替代结尾审核。相比事后与流式防护模型,本方法计算开销降低数量级,延迟几乎无增益。我们还提供实用部署方案,包括层选择、聚合策略、探测频率与触发阈值。最后,探针线性部分对应残差空间方向,支持检测与激活调控,成本极低。
原文摘要 · Abstract (English)
Deploying large language models in user-facing systems requires efficient output safety filtering. Existing approaches typically rely on a separate moderation model applied after generation, which doubles inference cost and only detects violations after generation completes. We observe that the signal needed for moderation is already present in the model hidden states. Based on this, we train lightweight token-level probes that operate directly on internal activations, producing per-token safety scores that can be aggregated for both offline evaluation and online intervention. The probe reuses activations from the generator and requires no additional forward pass, enabling sub millisecond per-token safety checks inside the decoding loop. A probe applied to a single mid layer recovers most decisions of a strong guard model, acting as a low cost surrogate optimized for latency rather than accuracy. In streaming settings, it can halt or modify unsafe outputs before they are fully generated, replacing end of sequence moderation with continuous token level monitoring. Compared to post hoc and streaming guard models, our method achieves orders of magnitude lower compute overhead with minimal latency cost. We also provide a practical deployment recipe, including layer selection, aggregation strategy, probing frequency, and triggering thresholds. Finally, we show that the probe linear component corresponds to a direction in residual space, enabling both detection and activation steering at negligible cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。