arXiv:2608.29098cs.AIcs.CV2026-08

构建150万条多模态安全数据集,实现五级风险分级评估。

SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models

论文配图:SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models
图 1 · 摘自论文原文
  • 构建五级有序评分体系,覆盖图像、请求、回复三层次判断。
  • 训练出8B模型在多基准上表现领先,F1提升约4%。
  • 数据集含15类危害与55个子类,适合安全检测与评测研究。

多模态安全审核需区分视觉内容、用户意图与助手行为带来的风险。现有防护机制通常针对单一目标训练,将安全评估简化为二元判断,导致风险难以跨交互比较,模糊案例被掩盖。本文提出SafeAtlas-VL,一个包含150万条训练样本的数据集,对图像、请求、响应三个层级进行五级有序评分。数据源自真实与合成来源,采用分歧感知标注流程,涵盖15种危害类别和55个细粒度子类别,覆盖广泛多模态安全场景。同时构建SafeAtlas-Bench,包含5000个保留实例,用于评估五级预测与连续风险得分。基于该数据集,通过目标条件微调训练SafeAtlas Guard系列模型,不仅支持五分类,还通过软累积序数头映射连续风险分数。实验表明,模型具备强泛化能力:即使不使用其他基准的训练集,也能在对应测试集上达到竞争力表现。特别地,8B模型整体表现最优,相比先前最先进方法,F1分数提升约4%。代码、数据与模型均已开源。警告:本文包含可能令人不适、有害、血腥或令人困扰的示例数据。

原文摘要 · Abstract (English)

Multimodal safety moderation requires distinguishing risks arising from visual content, user intent, and assistant behavior. Existing safeguards, however, are typically trained for a single judgment target and reduce safety assessment to a binary decision. Consequently, risk becomes difficult to compare across a multimodal interaction, and ambiguous cases are obscured. We introduce SafeAtlas-VL, a dataset of 1.5M training instances that places image-, request-, and response-level judgments on a five-level ordered scale. We curate a broad collection of safety-relevant data from both real-world and synthetic sources and apply a disagreement-aware annotation procedure. The resulting dataset spans 15 harm categories and 55 fine-grained subcategories, covering a broad range of multimodal safety scenarios. We also construct SafeAtlas-Bench, a held-out set of 5,000 instances for evaluating five-level predictions and continuous risk scores. Upon this dataset, we train the SafeAtlas Guard series of models via target-conditioned tuning for multimodal safety detection. Our models not only perform five-way classification of safety levels but also map safety to continuous scores through a soft cumulative ordinal head. Experimental results demonstrate that guard models trained on our dataset exhibit strong generalization: even without using the training sets of other benchmarks, they achieve competitive performance on the corresponding test sets. Notably, our 8B model attains the overall best performance, outperforming the previous SOTA by approximately 4% in F1 score. Code, data, and models are released to support further research. Warning: this paper contains example data that may be offensive, harmful, graphic, or disturbing.

多模态安全数据集风险评估大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。