用智能顾问模式解决大模型安全检查过严问题
Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs
- 安全检查改为生成风险提示+解释，不改变原模型行为
- 在20.8万数据集上训练，检测准确率高且减少误拒
- 推理开销低，适合实际部署的可信大模型系统
硬性安全过滤器常导致过度拒绝，且与模型规格不符；现有分类体系也忽视鲁棒性与诚实性，造成纸上安全却实用性差。本文提出软性安全顾问机制GaaA：由守护模型预测二元风险标签并生成简明解释，附加至原始查询前进行重推断，保持基座模型原规格运行。为支持训练与评估，构建了包含20.8万以上样本的多领域数据集GuardSet，涵盖有害与无害案例，并设计针对性鲁棒性与诚实性子集。通过监督微调与强化学习训练，使守卫模型实现标签与解释一致性。实验表明，该模型具备竞争力的检测准确率，且经其增强输入后响应质量优于原始提示。延迟分析显示，顾问推理仅消耗基模型5%以下计算资源，在真实有害输入率下端到端开销增加2-10%。整体上，GaaA有效引导模型遵循原规范，在保障安全的同时降低过严拒绝。
原文摘要 · Abstract (English)
Hard-gated safety checkers often over-refuse and misalign with a vendor's model spec; prevailing taxonomies also neglect robustness and honesty, yielding safer-on-paper yet less useful systems. This work introduces Guardian-as-an-Advisor (GaaA), a soft-gating pipeline where a guardian predicts a binary risk label plus a concise explanation and prepends this advice to the original query for re-inference, keeping the base model operating under its original spec. To support training and evaluation, GuardSet is constructed, a 208k+ multi-domain dataset unifying harmful and harmless cases with targeted robustness and honesty slices. GuardAdvisor is trained via SFT followed by RL to enforce label-explanation consistency. GuardAdvisor attains competitive detection accuracy while enabling the advisory workflow; when used to augment inputs, responses improve over unaugmented prompts. A latency study shows advisor inference uses below 5% of base-model compute and adds only 2-10% end-to-end overhead under realistic harmful-input rates. Overall, GaaA steers models to comply with the model spec, maintaining safety while reducing over-refusal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。