提出可识别视觉目标的安检机制,防止多模态模型被误导执行危险操作。
COMIC: Reference-Aware Safety Gating for Multimodal Large Language Models

- 通过分析操作与视觉目标的绑定关系,识别潜在安全风险。
- 在多种攻击测试中显著提升模型鲁棒性,误拦率低于5%。
- 适合需要高安全性的文档、截图处理场景,如医疗或金融应用。
多模态大语言模型(MLLMs)越来越多用于处理截图、扫描文档、图表等视觉输入。这种使用方式引入了新安全风险:许多多模态越狱攻击中,提示词和图像单独看均无害,但当模型将看似正常的操作(如摘要、翻译、遵循)绑定到特定视觉目标时,才会触发不安全行为。这暴露出当前防御机制的结构性缺陷——大多对提示-图像整体进行审查,而真正的安全相关单元是解引用后生成的“操作-目标对”。本文识别并分析了这一依赖视觉参考的失效模式,发现现有防御在有害语义局部化、仅在接地后激活、依赖视觉参考解析时性能下降。为此,我们提出COMIC(上下文-操作-模态-图像分类器),一种面向MLLM的参考感知预生成安全闸门。COMIC首先推断请求操作与参考类型,从OCR和开放词汇提议中构建候选目标,定位合理指代对象,并对显式的操作-目标对进行安全评估。为保守处理歧义,采用最大风险聚合结合质量感知路由,在决定放行或拦截前进行综合判断。我们在多个开源MLLM上评估COMIC,覆盖局部化与广义多模态越狱基准以及良性参考敏感设置。结果表明,COMIC持续提升鲁棒性,同时保持良好实用性和效率。更广泛而言,我们的研究暗示:若不建模所请操作、其作用的视觉目标及接地置信度,多模态安全无法可靠保障。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) are increasingly used to interact with screenshots, scanned documents, diagrams, and other visually grounded inputs. This shift introduces a new safety risk: in many multimodal jailbreaks, neither the prompt nor the image is harmful in isolation. Unsafe behavior emerges only when the model binds an apparently benign operation, such as summarizing, translating, or following, to a localized visual target. This reveals a structural weakness in current multimodal defenses, which largely moderate the prompt-image pair as a whole even though the true security-relevant unit is the grounded operation-target pair produced during dereference. In this work, we identify and analyze this reference-dependent failure mode and show that existing defenses degrade when harmful semantics are localized, activated only after grounding, and dependent on visual reference resolution. To address this problem, we propose COMIC (Context-Operation-Modality-Image-Classifier), a reference-aware pre-generation safety gate for MLLMs. COMIC first infers the requested operation and reference type, constructs candidate targets from OCR and open-vocabulary proposals, grounds plausible referents, and evaluates safety over explicit operation-target pairs. To handle ambiguity conservatively, COMIC combines max-risk aggregation with quality-aware routing before deciding whether to forward or block a request. We evaluate COMIC across multiple open-source MLLMs, localized and broader multimodal jailbreak benchmarks, and benign reference-sensitive settings. The results show that COMIC consistently improves robustness while preserving benign utility and practical efficiency. More broadly, our findings suggest that multimodal safety cannot be enforced reliably without modeling the requested operation, the visual target to which it applies, and the confidence of that grounding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。