arXiv:2511.12497cs.CLcs.AI2025-11被引 3

轻量级AI安全护栏,能识别有害内容和对抗性提示。

SGuard-v1: Safety Guardrail for Large Language Models

  • 分两模块:内容过滤与越狱检测,专攻安全风险
  • 覆盖60类攻击,在140万数据上训练,误判率低
  • 支持12种语言,开源可用,适合部署于对话系统

我们提出 SGuard-v1,一种针对大语言模型的轻量级安全护栏,包含两个专用模型:ContentFilter 负责依据 MLCommons 危险分类框架检测提示与回复中的安全风险;JailbreakFilter 基于精心设计的课程学习策略,在整合数据集和已有研究基础上训练,覆盖60种主要对抗性提示攻击,同时减少误判。SGuard-v1 基于 2B 参数的 Granite-3.3-2B-Instruct 模型,支持12种语言,使用约140万条真实与合成数据进行指令微调,并按功能分配至两个组件。在公开与私有安全基准上评估,SGuard-v1 实现了领先的安全性能,且部署开销小。其多类别安全预测与二值置信度评分提升了下游应用的可解释性。项目以 Apache-2.0 许可证开源,促进安全研究与实际部署。

原文摘要 · Abstract (English)

We present SGuard-v1, a lightweight safety guardrail for Large Language Models (LLMs), which comprises two specialized models to detect harmful content and screen adversarial prompts in human-AI conversational settings. The first component, ContentFilter, is trained to identify safety risks in LLM prompts and responses in accordance with the MLCommons hazard taxonomy, a comprehensive framework for trust and safety assessment of AI. The second component, JailbreakFilter, is trained with a carefully designed curriculum over integrated datasets and findings from prior work on adversarial prompting, covering 60 major attack types while mitigating false-unsafe classification. SGuard-v1 is built on the 2B-parameter Granite-3.3-2B-Instruct model that supports 12 languages. We curate approximately 1.4 million training instances from both collected and synthesized data and perform instruction tuning on the base model, distributing the curated data across the two component according to their designated functions. Through extensive evaluation on public and proprietary safety benchmarks, SGuard-v1 achieves state-of-the-art safety performance while remaining lightweight, thereby reducing deployment overhead. SGuard-v1 also improves interpretability for downstream use by providing multi-class safety predictions and their binary confidence scores. We release the SGuard-v1 under the Apache-2.0 License to enable further research and practical deployment in AI safety.

大模型安全内容过滤对抗防御轻量部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。