利用大模型内部特征提升有害内容检测效率与精度
LLM Safety From Within: Detecting Harmful Content with Internal Representations
- 通过线性探测识别关键安全神经元，融合多层内部表示
- 比现有开源模型性能更优，参数量少250倍
- 支持实时流式检测，适合生产环境部署
防护模型广泛用于检测用户提示和大模型输出中的有害内容。然而，当前最先进的防护模型仅依赖末端层表示，忽略了分布在内部各层的安全相关特征。本文提出SIREN，一种轻量级防护模型，充分利用这些内部特征。通过线性探测识别安全神经元，并采用自适应层加权策略进行融合，SIREN在不修改底层模型的前提下构建有害性检测器。全面评估显示，SIREN在多个基准上显著优于现有开源防护模型，同时仅需250倍更少的可训练参数。此外，SIREN在未见基准上表现出更强泛化能力，天然支持实时流式检测，相比生成式防护模型显著提升推理效率。总体而言，结果表明大模型内部状态是实现高效、高性能有害内容检测的有力基础。
原文摘要 · Abstract (English)
Guard models are widely used to detect harmful content in user prompts and LLM responses. However, state-of-the-art guard models rely solely on terminal-layer representations and overlook the rich safety-relevant features distributed across internal layers. We present SIREN, a lightweight guard model that harnesses these internal features. By identifying safety neurons via linear probing and combining them through an adaptive layer-weighted strategy, SIREN builds a harmfulness detector from LLM internals without modifying the underlying model. Our comprehensive evaluation shows that SIREN substantially outperforms state-of-the-art open-source guard models across multiple benchmarks while using 250 times fewer trainable parameters. Moreover, SIREN exhibits superior generalization to unseen benchmarks, naturally enables real-time streaming detection, and significantly improves inference efficiency compared to generative guard models. Overall, our results highlight LLM internal states as a promising foundation for practical, high-performance harmfulness detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。