让视觉语言模型像用工具一样一步步推理安全判断,提升防攻击和回应质量。
SaFeR-ToolKit: Structured Reasoning via Virtual Tool Calling for Multimodal Safety
- 设计可追踪的工具调用流程,分步验证视觉与意图是否安全
- 在30亿和70亿参数模型上,安全得分超84%,回应更合理且不滥拒
- 适合研究模型安全、对齐机制或想提升多模态系统可靠性的团队
视觉语言模型仍易受多模态越狱攻击和过度拒绝影响,因安全判断需同时依赖视觉证据与用户意图,而多数对齐方法仅监督最终回答。为此,我们提出SaFeR-ToolKit,将安全决策形式化为可核查的协议:规划器定义角色、感知→推理→决策工具链及受限转移图;响应者在最终答案前输出带类型的关键值工具调用轨迹。为确保协议在实践中被可靠遵循,采用三阶段课程训练单策略(SFT→DPO→GRPO),其中GRPO直接监督工具使用,而非仅基于答案反馈。贡献有二:一、构建首个基于工具的安全推理数据集,含31,654条样本(SFT 6k,DPO 18.6k,GRPO 6k)及1k保留评估集;二、实验显示,在Qwen2.5-VL上,该方法显著提升安全性/帮助性/推理严谨性(3B:29.39/45.04/4.98 → 84.40/71.13/78.87;7B:53.21/52.92/19.26 → 86.34/80.79/85.34),同时保持通用能力(3B:58.67 → 59.21;7B:66.39 → 66.81)。代码已开源。
原文摘要 · Abstract (English)
Vision-language models remain susceptible to multimodal jailbreaks and over-refusal because safety hinges on both visual evidence and user intent, while many alignment pipelines supervise only the final response. To address this, we present SaFeR-ToolKit, which formalizes safety decision-making as a checkable protocol. Concretely, a planner specifies a persona, a Perception $\to$ Reasoning $\to$ Decision tool set, and a constrained transition graph, while a responder outputs a typed key-value tool trace before the final answer. To make the protocol reliably followed in practice, we train a single policy with a three-stage curriculum (SFT $\to$ DPO $\to$ GRPO), where GRPO directly supervises tool usage beyond answer-level feedback. Our contributions are two-fold: I. Dataset. The first tool-based safety reasoning dataset, comprising 31,654 examples (SFT 6k, DPO 18.6k, GRPO 6k) plus 1k held-out evaluation. II. Experiments. On Qwen2.5-VL, SaFeR-ToolKit significantly improves Safety/Helpfulness/Reasoning Rigor on 3B (29.39/45.04/4.98 $\to$ 84.40/71.13/78.87) and 7B (53.21/52.92/19.26 $\to$ 86.34/80.79/85.34), while preserving general capabilities (3B: 58.67 $\to$ 59.21; 7B: 66.39 $\to$ 66.81). Codes are available at https://github.com/Duebassx/SaFeR_ToolKit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。