SafeVision通过动态推理实现高效安全图像过滤,支持政策自适应与解释。
SafeVision: Efficient Image Guardrail with Robust Policy Adherence and Explainability
- 引入类人推理机制,动态对齐安全策略,无需重新训练
- 在VisionHarm-T/C上分别超越GPT-4o 8.6%和15.5%,速度超16倍
- 内置可解释性与新数据集,适合需要透明合规的AI应用
随着数字媒体的迅速普及,高效透明的内容安全防护需求愈发迫切。传统图像安全模型受限于预定义类别,仅依赖特征学习而缺乏语义推理,常出现误判,且难以应对新威胁,需昂贵重训。为此,我们提出SafeVision,一种融合人类式推理的新型图像安全防护系统。其包含高效数据收集与生成框架、策略遵循训练流程及定制损失函数,并设计多样化QA生成与训练策略以提升学习效果。SafeVision可在推理时动态对齐演化中的安全政策,避免重训,确保精准风险评估与可解释性。针对现有危险图像基准数据集粒度不足或覆盖有限的问题,我们构建了高质数据集VisionHarm,包含VisionHarm-T(第三方)与VisionHarm-C(综合版)两个子集,涵盖多样有害类别。大量实验表明,SafeVision在多个基准上达领先性能:在VisionHarm-T上比GPT-4o高8.6%,在VisionHarm-C上高15.5%,同时速度超过16倍。
原文摘要 · Abstract (English)
With the rapid proliferation of digital media, the need for efficient and transparent safeguards against unsafe content is more critical than ever. Traditional image guardrail models, constrained by predefined categories, often misclassify content due to their pure feature-based learning without semantic reasoning. Moreover, these models struggle to adapt to emerging threats, requiring costly retraining for new threats. To address these limitations, we introduce SafeVision, a novel image guardrail that integrates human-like reasoning to enhance adaptability and transparency. Our approach incorporates an effective data collection and generation framework, a policy-following training pipeline, and a customized loss function. We also propose a diverse QA generation and training strategy to enhance learning effectiveness. SafeVision dynamically aligns with evolving safety policies at inference time, eliminating the need for retraining while ensuring precise risk assessments and explanations. Recognizing the limitations of existing unsafe image benchmarks, which either lack granularity or cover limited risks, we introduce VisionHarm, a high-quality dataset comprising two subsets: VisionHarm Third-party (VisionHarm-T) and VisionHarm Comprehensive(VisionHarm-C), spanning diverse harmful categories. Through extensive experiments, we show that SafeVision achieves state-of-the-art performance on different benchmarks. SafeVision outperforms GPT-4o by 8.6% on VisionHarm-T and by 15.5% on VisionHarm-C, while being over 16x faster. SafeVision sets a comprehensive, policy-following, and explainable image guardrail with dynamic adaptation to emerging threats.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。