arXiv:2604.00310cs.LGcs.AI2026-04

用内部注意力机制提前识别有害多模态输入,提升模型安全性。

CASA: Classification Augmented with Safety Attention for Robust Multimodal Alignment

论文配图:CASA: Classification Augmented with Safety Attention for Robust Multimodal Alignment
图 1 · 摘自论文原文
  • 通过安全注意力机制在生成前判断查询是否危险,仅需一次内部决策。
  • 在7种攻击类型上平均成功率降低超90%,且不依赖外部分类器。
  • 适合需要强安全性的多模态大模型应用,如图文对话、语音助手。

多模态大语言模型(MLLMs)在面对有害查询时,常因跨模态交互导致安全对齐失效。仅在文本上训练的模型在扩展至多模态后,攻击成功率显著上升。本文提出CASA(Classification Augmented with Safety Attention),一种基于内部表示的条件解码策略,在生成响应前通过安全注意力机制计算专用查询提示与输入之间的注意力,以预测二元安全标记。该方法无需外部分类器或附加头,也无需针对特定模态的安全微调。相比4个基线模型,在MM-SafetyBench、JailbreakV-28k及对抗性音频测试等基准上,搭配标准安全提示时,平均攻击成功率降低超过90%;仅使用其内部安全机制时,降低54%–61%。实证表明,CASA在良性输入上仍保持良好性能,经13名训练标注员的人工评估验证。结果证明CASA是一种简单且通用的多模态大模型安全增强框架。

原文摘要 · Abstract (English)

Multimodal large-language models (MLLMs) often experience degraded safety alignment when harmful queries exploit cross-modal interactions. Models aligned on text alone show a higher rate of successful attacks when extended to two or more modalities. We propose a simple conditional decoding strategy, CASA (Classification Augmented with Safety Attention) that uses internal representations of MLLMs to predict a binary safety token before response generation. The classification is guided by a safety attention mechanism that computes attention between a dedicated query prompt and the input to scale the classification logits, sharpening the model's ability to detect malicious queries. Our design ensures robust safety alignment without relying on any external classifier or auxiliary head, and without modality-specific safety fine-tuning. Unlike prior safety-decoding methods that attach an external classifier or moderation head, or rescale every generated token with a reward model, CASA makes a single, internal, attention-gated safety decision over the shared multimodal representation, which is precisely what lets safety learned from text-only training transfer to image and audio inputs. Compared against 4 baseline models across 7 attack types spanning text, image, and audio modalities, evaluated on benchmarks such as MM-SafetyBench, JailbreakV-28k, and adversarial audio tests, CASA lowers the average attack success rate by >90% relative when paired with a standard safety prompt (applied to all baselines for fair comparison), and by 54--61% using its internal safety mechanism alone. Our empirical evaluations also show that CASA~maintains strong utility on benign inputs, validated through both automated and human evaluations (via 13 trained annotators). Together, these results highlight CASA as a simple and generalizable framework to improve multimodal LLM safety.

多模态安全注意力机制大模型对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。