arXiv:2606.22873cs.CVcs.CL2026-06被引 2

让AI安全审查能实时适应新规,动态判断多模态内容风险

SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning

论文配图:SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning
图 1 · 摘自论文原文
  • 将安全政策作为实时输入,逐条比对规则判断内容是否违规
  • 在56,340个案例上实现各任务最优F1,政策变更时准确率提升至74.15%
  • 适合需要灵活合规的医疗、金融等多模态应用部署场景

视觉语言模型在消费、医疗、金融和企业应用中日益普及,其安全风险涵盖多模态问答、助手回复及跨模态组合,且审核政策随产品、地区和部署阶段变化。现有防护机制多依赖固定分类体系或仅覆盖有限交互场景,难以适应运行时政策调整。本文提出SingGuard,一种可适配策略的多模态防护模型家族,将当前安全策略以自然语言输入,逐条核验目标内容并输出安全标签与触发规则。为兼顾效率与可解释性,SingGuard支持从直接判断到基于策略推理的多种推理模式。我们还引入SingGuard-Bench,包含56,340个样本的多模态防护基准,覆盖80余种细粒度风险类型,涵盖跨模态联合风险(单模态无害但组合后隐含危险意图)。在六个基准族(35个数据集)中,SingGuard在每类均达到最先进平均F1;动态策略评估显示,政策变更时遵循准确率从0.6465提升至0.7415。代码已开源。

原文摘要 · Abstract (English)

Vision-language models (VLMs) are increasingly deployed in consumer, medical, financial, and enterprise applications. This broad deployment expands the safety surface: risks can arise from multimodal question answering, assistant responses, and cross-modal composition, while moderation policies may vary across products, regions, and deployment stages. Most existing guardrails either rely on fixed taxonomies or target only a narrow set of interaction settings, which limits their adaptability when safety rules change at deployment time. We present \textbf{SingGuard}, a policy-adaptive multimodal guardrail model family for safety assessment in multimodal conversations. SingGuard treats the active policy as a runtime input: given natural-language rules, it checks the target content against the active policy rule by rule and predicts both the safety label and the triggered rule. To balance efficiency and interpretability, SingGuard supports fast, hybrid, and slow inference regimes along a fast-to-slow reasoning spectrum, ranging from direct safety judgments to policy-grounded deliberation. We further optimize this behavior with fast--slow decoupled reinforcement learning. We also introduce \textbf{SingGuard-Bench}, a multimodal guardrail benchmark with 56{,}340 examples spanning 80+ fine-grained risk types across multimodal QA, adversarial attack, and dynamic-rule evaluation settings, including cross-modal joint-risk cases where each modality is harmless in isolation but their composition implies unsafe intent. Across six benchmark families (35 datasets), SingGuard achieves state-of-the-art average F1 in every family. Dynamic-rule evaluation further shows improved policy-following accuracy from 0.6465 to 0.7415 under runtime policy shifts. Our code is available at https://github.com/inclusionAI/Sing-Guard.

多模态安全动态规则大模型防护智能审核

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。