检测多模态推理中的隐性风险,实时拦截有害中间思考过程。
GuardTrace-VL: Detecting Unsafe Multimodel Reasoning via Iterative Safety Supervision
- 通过联合图文分析监控完整推理链,识别中间阶段的不安全内容。
- 在跨域测试集上达到93.1%的F1分数,比现有方法提升13.5%。
- 适合关注多模态模型安全部署的研究者与开发者使用。
多模态大推理模型(MLRMs)在生成显式中间推理链的任务中日益普及。然而,即使最终答案无害,推理过程仍可能包含不当内容,带来部署风险。现有多模态安全防护主要评估输入问题和最终答案,忽视中间推理阶段,导致偏见推断或违反政策的视觉上下文滥用等问题难以被发现。本文提出GuardTrace-VL,一种基于视觉感知的安全审计器,通过联合图像-文本分析对完整的问答-思考-答案(QTA)流程进行监控,实现在推理阶段即时检测不安全内容。为支持训练与评估,我们构建了GuardTrace数据集,采用多样化提示策略生成,并经由多模态大模型与人工投票验证进行精炼。此外,我们提出三阶段渐进式训练方案,结合数据精炼流程,使模型能学习不同风险等级下的细微、上下文依赖的安全偏好。在涵盖域内与域外场景的测试集上,GuardTrace-VL模型在不安全推理检测任务中取得93.1%的F1分数,较此前最强的多模态安全防御方法提升13.5%。代码将公开可用。
原文摘要 · Abstract (English)
Multimodal large reasoning models (MLRMs) are increasingly deployed for vision-language tasks that produce explicit intermediate rationales. However, reasoning traces can contain unsafe content even when the final answer is non-harmful, creating deployment risks. Existing multimodal safety guards primarily evaluate only the input question and the final answer, neglecting the intermediate reasoning process. This oversight allows undetected harm, such as biased inferences or policy-violating use of visual context, to emerge during reasoning. We introduce GuardTrace-VL, a vision-aware safety auditor that monitors the full Question-Thinking-Answer (QTA) pipeline via joint image-text analysis, enabling detection of unsafe content as it emerges in the reasoning stage. To support training and evaluation, we construct the GuardTrace dataset, which is generated through diverse prompting strategies and refined via a MLRM- and human-based voting and verification pipeline. Furthermore, we propose a three-stage progressive training scheme combined with the data refinement process, enabling the model to learn nuanced and context-dependent safety preferences according to different risk levels. On our proposed test set covering both in-domain and out-of-domain scenarios, GuardTrace-VL model achieves an F1 score of 93.1% on unsafe reasoning detection tasks, representing a 13.5% improvement in F1 score compared to the previous strongest multimodal safety defense methods. The codes will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。