提出可自适应安全策略的图像防护方法,解决模型过拟合固定政策的问题。
Towards Policy-Adaptive Image Guardrail: Benchmark and Method
- 用图像编辑生成政策对齐数据,构建跨策略评估基准
- 引入基于可验证奖励的强化学习,实现策略动态适配
- 适合需要持续更新安全规则的AI内容审核场景
准确识别敏感或有害视觉内容(即有害图像防护)在诸多应用场景中至关重要。该任务需持续适应不断变化的安全策略、不同领域及时间演进。然而,传统分类器受限于固定类别,新政策出台时需频繁重新训练。视觉语言模型(VLMs)为动态安全防护提供了更灵活、泛化性更强的基础。尽管如此,现有基于VLM的防护方法通常仅在单一固定安全政策下训练与评估。我们发现这些模型严重过拟合于已见政策,无法泛化至未见政策,甚至丧失基本指令遵循能力与通用知识。本文作出两项关键贡献:首先,提出SafeEditBench,一个新型评估套件,通过图像编辑模型将有害图像转换为对应安全版本,生成政策对齐数据集;每对安全-非安全图像在视觉上仅局部区域违反特定安全规则,人类标注者据此提供五种不同政策下的精准标签,实现细粒度跨政策泛化评估。其次,提出SafeGuard-VL,一种基于可验证奖励的强化学习方法(RLVR),不依赖固定政策的监督微调,而是通过政策驱动的奖励显式优化模型,促进在演化政策中的鲁棒适应。大量实验验证了该方法在多种政策下防护无效图像的有效性。
原文摘要 · Abstract (English)
Accurate rejection of sensitive or harmful visual content, i.e., harmful image guardrail, is critical in many application scenarios. This task must continuously adapt to the evolving safety policies and content across various domains and over time. However, traditional classifiers, confined to fixed categories, require frequent retraining when new policies are introduced. Vision-language models (VLMs) offer a more adaptable and generalizable foundation for dynamic safety guardrails. Despite this potential, existing VLM-based safeguarding methods are typically trained and evaluated under only a fixed safety policy. We find that these models are heavily overfitted to the seen policy, fail to generalize to unseen policies, and even lose the basic instruction-following ability and general knowledge. To address this issue, in this paper we make two key contributions. First, we benchmark the cross-policy generalization performance of existing VLMs with SafeEditBench, a new evaluation suite. SafeEditBench leverages image-editing models to convert unsafe images into safe counterparts, producing policy-aligned datasets where each safe-unsafe image pair remains visually similar except for localized regions violating specific safety rules. Human annotators then provide accurate safe/unsafe labels under five distinct policies, enabling fine-grained assessment of policy-aware generalization. Second, we introduce SafeGuard-VL, a reinforcement learning-based method with verifiable rewards (RLVR) for robust unsafe-image guardrails. Instead of relying solely on supervised fine-tuning (SFT) under fixed policies, SafeGuard-VL explicitly optimizes the model with policy-grounded rewards, promoting verifiable adaptation across evolving policies. Extensive experiments verify the effectiveness of our method for unsafe image guardrails across various policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。