通过前瞻推理与安全优化，减少流式安全检测中的误拒和漏洞。
FreoStream:Enhancing Stream Guardrails via Future-Aware Reasoning and Safety-Aligned Optimization
- 引入未来感知推理机制，结合上下文判断是否真有害。
- 在多个基准上实现更低误拒率与更强的越狱防御能力。
- 适合需要实时安全过滤且避免误伤的AI应用开发者。
流式安全防护可在完整回复生成前进行逐标记级的安全检测。然而，现有方法常因缺乏完整上下文而过度保守，导致对安全但敏感的标记错误拒绝（过拒），且难以识别隐含恶意的越狱内容。为此，本文提出FreoStream，一种新型流式安全防护框架。具体地，当基础防护模型检测到不安全标记时，FreoStream微调一个LoRA模块，执行未来感知推理。该推理遵循‘预测未来—推理上下文—最终判断’范式，通过引入未来信息有效缓解过拒问题。此外，引入安全对齐优化模块，从推理梯度中提取安全对齐成分，用于更新基础防护模型，从而提升流式安全检测能力。在多个安全基准上的大量实验表明，FreoStream相比现有流式防护方法，显著降低过拒率并增强越狱防御能力。
原文摘要 · Abstract (English)
Stream guardrails enable token-level safety detection before full responses are generated. However, they often make overly conservative judgements and block those sensitive but safe tokens, which is known as over-refusal. Due to lack of full context, they also fail to detect implicitly harmful content from jailbreaking. To address these challenges, we propose FreoStream, a novel streaming guardrail framework. Specifically, FreoStream fine-tunes a LoRA module to perform Future-Aware Reasoning when the base guardrail detects unsafe tokens. The reasoning process follows a Future-Reason-Judge paradigm: predict the future, reason about the full context and give the final judgement. This design can effectively reduce over-refusal by incorporating the future information. Moreover, we introduce the Safety-Aligned Optimization module that extracts the safety-aligned component from the reasoning gradients to update the base guardrail model, thereby enhancing streaming safety detection. Extensive experiments on various safety benchmarks demonstrate that FreoStream achieves lower over-refusal rates and better jailbreak defense compared to existing streaming guardrails.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。