不靠推理也能高效拦截有害内容，实时视频语言安全防护新方案
When Are Reasoning-Based Guardrails Not Efficient? ResponseGuard: A Fast Vision-Language Guard for Real-Time Moderation
- 单次前向传播提取请求、回复和图像的联合表示，直接判断是否有害
- 20亿参数模型比30亿参数推理型模型快150倍，检测准确率更高
- 适合需要低延迟响应的实时内容审核场景，如直播或即时对话
视觉-语言大模型生成答案时以词元流形式输出，因此安全防护机制必须跟上生成速度，在用户读取前阻止有害内容。现有防护方法通过生成思维链再做出判断，认为逐步推理更安全，但导致模型解码大量词元，造成延迟高、计算重。本文质疑：视觉-语言防护是否真需推理？提出无需思维链的ResponseGuard，仅用一次前向传播，从请求、回复与图像的联合池化表示中直接输出有害性判断。在标准多模态防护基准上，20亿参数的ResponseGuard在回复有害性检测上超越30亿参数的推理型模型，且时间成本降低约150倍。在请求有害性检测上，推理型模型仍占优，但差距集中在仅含图像的测试项。分析发现，二者均使用冻结的视觉编码器，且推理型模型几乎未关注图像信息。基于单次检测，ResponseGuard可逐句实时筛查生成流，及时中断有害回答。研究表明，对视觉-语言模型的响应防护，单次校准标签已足够提供安全信号。代码、模型与数据集已开源。
原文摘要 · Abstract (English)
A vision-language AI assistant returns its answer as a stream of generated tokens. Therefore, a safety guard that watches that answer has to keep up with the stream and stop a harmful reply before a user reads it. Recent vision-language guardrails instead generate a chain of thought before they issue a verdict. They believe that step-by-step reasoning yields a safer guard. This design makes the guard heavy and slow, since the model must decode many tokens for harmfulness detection. We pose the question of whether a vision-language guard really needs to reason in order to screen a response. We answer with a guard that has no chain. ResponseGuard reads a harmful verdict from a single pooled representation of the request, the response, and the image in one forward pass. Across a standard multimodal guardrail benchmark, our 2B ResponseGuard outperforms a recent 3B reasoning-based vision-language guard on response harmfulness detection, without any reasoning and at about 150 times lower time cost. On request harmfulness the reasoning guard retains an overall lead, and the remaining gap on both tracks sits on the image-only cells. We observe that the gap may stem from the frozen vision encoders that both designs use rather than from the missing chain. We have also found the reasoning guard directs almost none of its verdict attention to the image. Based on a single-pass detection, ResponseGuard can screen an answer sentence by sentence as it streams and stop a harmful answer before it finishes. For guarding the response of a vision-language model, a calibrated single-pass label may provide a sufficient safety signal. We fully release all source code, trained models, and datasets at https://github.com/ndb796/ResponseGuard.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。