让大模型同时思考安全与风险,更准更透明地拦截有害请求
A Dual-Hypothesis Reasoning Framework for LLM Guardrails

- 双假设推理:先看提示的正反两种可能解释再判断
- 用轻量级训练方法实现比昂贵方案更好的安全检测效果
- 能给出具体依据说明为何判定为不安全,适合需要可解释性的场景
我们提出ARBITER,一种新型大模型安全防护框架,包含两项关键创新:(i) 双假设推理,即在做安全决策前显式考虑提示的合理与不合理两种解释;(ii) 多组件监督微调(MC-SFT),将模型输出分解为逻辑组件并按重要性加权。现有基于推理的安全防护通常依赖昂贵流程,如使用大型或闭源教师模型生成推理链,以及全参数微调。相比之下,ARBITER采用低成本自生成推理链和基于LoRA的高效微调,在性能上仍优于这些高成本方法。此外,ARBITER能提供可信的证据短语解释,增强决策透明性。在三个安全审核基准上的实验表明,ARBITER在各类评估中均优于现有基于推理与非推理的基线方法,尤其在跨域测试中表现突出。
原文摘要 · Abstract (English)
We propose ARBITER, a novel LLM guardrail framework that introduces two key ideas: (i) dual-hypothesis reasoning, a reasoning method for LLM guardrails that explicitly considers both safe and unsafe interpretations of a prompt before making a safety decision, and (ii) multi-component supervised fine-tuning (MC-SFT), a structured training loss for reasoning-based guardrails that decomposes LLM outputs into logical components and weights them according to their importance. Existing reasoning-based guardrails often rely on expensive procedures, such as generating reasoning traces using larger or closed-source teacher models and applying full-parameter fine-tuning. In contrast, ARBITER uses a cost-effective self-generation strategy for reasoning traces and LoRA-based parameter-efficient fine-tuning while still achieving better performance than these expensive approaches. Additionally, ARBITER provides faithful evidence-phrase explanations for unsafe decisions, enabling a more transparent and interpretable guardrail method. Experiments on three safety moderation benchmarks show that ARBITER outperforms existing reasoning-based and non-reasoning guardrail baselines, with clear gains in out-of-domain evaluations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。