Kelp实现实时风险检测,用隐状态动态追踪生成内容中的潜在危害。
Kelp: A Streaming Safeguard for Large Models via Latent Dynamics-Guided Risk Detection
- 通过中间隐藏状态建模风险演化,实现生成过程中的流式风险检测。
- 在多个模型上平均F1提升15.61%,仅增加0.5毫秒每词延迟。
- 适合需要低延迟实时内容安全的场景,如对话系统、多模态生成。
大型模型(LMs)虽具强大生成能力,但其开放性可能引发有害或偏见内容。现有防护机制多为事后检测,存在内容暴露风险,且受延迟约束常采用轻量模型,影响检测精度。本文提出Kelp,一种可插拔的流式风险检测框架,利用中间隐藏状态构建流式隐状态动态头(SLD),建模生成序列中风险的时序演化,实现更精准的实时检测。为保障实际应用中的可靠流式审核,引入锚定时间一致性(ATC)损失,通过良-害时序先验强制预测单调性。此外,构建StreamGuardBench——一个基于模型的基准测试,支持受保护模型的实时响应,覆盖文本与视觉语言任务的真实流式场景。在多种模型与数据集上,Kelp持续优于最先进事后防护机制及先前插件探测器,平均F1提升15.61%,仅需2000万参数,每词额外延迟低于0.5毫秒。
原文摘要 · Abstract (English)
Large models (LMs) are powerful content generators, yet their open-ended nature can also introduce potential risks, such as generating harmful or biased content. Existing guardrails mostly perform post-hoc detection that may expose unsafe content before it is caught, and the latency constraints further push them toward lightweight models, limiting detection accuracy. In this work, we propose Kelp, a novel plug-in framework that enables streaming risk detection within the LM generation pipeline. Kelp leverages intermediate LM hidden states through a Streaming Latent Dynamics Head (SLD), which models the temporal evolution of risk across the generated sequence for more accurate real-time risk detection. To ensure reliable streaming moderation in real applications, we introduce an Anchored Temporal Consistency (ATC) loss to enforce monotonic harm predictions by embedding a benign-then-harmful temporal prior. Besides, for a rigorous evaluation of streaming guardrails, we also present StreamGuardBench-a model-grounded benchmark featuring on-the-fly responses from each protected model, reflecting real-world streaming scenarios in both text and vision-language tasks. Across diverse models and datasets, Kelp consistently outperforms state-of-the-art post-hoc guardrails and prior plug-in probes (15.61% higher average F1), while using only 20M parameters and adding less than 0.5 ms of per-token latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。