通过两阶段优化提升视觉语言模型抗越狱攻击能力，兼顾安全与视觉理解。
Activation Steering Meets Preference Optimization: Defense Against Jailbreaks in Vision Language Models
- 分两阶段：先自适应生成层特定干扰向量，再用偏好优化精炼结果。
- 在多个攻击场景下显著提升安全性，同时保持原有视觉理解性能。
- 适合关注大模型安全、对抗防御的研究者和应用开发者。
视觉语言模型（VLM）在融合视觉与文本信息进行理解和推理方面表现出色，但极易受到对抗攻击。尽管激活值引导被视为有前景的防御方法，现有方案常依赖任务特定的对比提示提取有害方向，存在性能不足且损害视觉定位能力的问题。为此，我们提出面向VLM的序列级偏好优化框架（SPO-VLM），采用两阶段设计：第一阶段从多样化数据源计算自适应层特定的引导向量，实现推理时对有害行为的泛化抑制；第二阶段通过序列级偏好优化，结合自动化毒性评估与基于图文一致性的奖励信号，实现安全且语义连贯的文本生成。该结构在保证轻量级干预的基础上，实现深度策略优化，平衡效率与效果。大量实验表明，SPO-VLM在激活引导与偏好优化双重机制下显著增强模型安全性，同时在良性任务上保持优异性能，不牺牲视觉理解能力。代码、模型权重与评估工具包将公开，以支持可复现性与后续研究。警告：本文可能包含冒犯性或有害文本与图像。
原文摘要 · Abstract (English)
Vision Language Models (VLMs) have demonstrated impressive capabilities in integrating visual and textual information for understanding and reasoning, but remain highly vulnerable to adversarial attacks. While activation steering has emerged as a promising defence, existing approaches often rely on task-specific contrastive prompts to extract harmful directions, which exhibit suboptimal performance and can degrade visual grounding performance. To address these limitations, we propose \textit{Sequence-Level Preference Optimization} for VLM (\textit{SPO-VLM}), a novel two-stage defense framework that combines activation-level intervention with policy-level optimization to enhance model robustness. In \textit{Stage I}, we compute adaptive layer-specific steering vectors from diverse data sources, enabling generalized suppression of harmful behaviors during inference. In \textit{Stage II}, we refine these steering vectors through a sequence-level preference optimization process. This stage integrates automated toxicity assessment, as well as visual-consistency rewards based on caption-image alignment, to achieve safe and semantically grounded text generation. The two-stage structure of SPO-VLM balances efficiency and effectiveness by combining a lightweight mitigation foundation in Stage I with deeper policy refinement in Stage II. Extensive experiments shown SPO-VLM enhances safety against attacks via activation steering and preference optimization, while maintaining strong performance on benign tasks without compromising visual understanding capabilities. We will release our code, model weights, and evaluation toolkit to support reproducibility and future research. \textcolor{red}{Warning: This paper may contain examples of offensive or harmful text and images.}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。