arXiv:2602.02557cs.LGcs.AI2026-02

更强的图文对齐反而让文本攻击更容易侵入音频模型,带来隐藏安全风险。

The Alignment Curse: Modality Alignment Supercharges Audio Attacks via Text Transfer

  • 通过分析图文对齐与跨模态攻击转移的关系,揭示安全隐患机制。
  • 文本攻击转移至音频后效果不逊于原生音频攻击,甚至更优。
  • 适用于评估多模态模型安全性的研究者及开发者参考。

端到端训练的全模态模型通过强化文本-音频对齐显著提升了音频能力,但这种对齐是否无意中促进了安全漏洞在模态间的传播仍缺乏研究。由于基于文本的越狱攻击技术远比音频攻击成熟,若其可系统性迁移,则当前音频安全评估可能低估了源自文本模态的风险。本文提出「对齐诅咒」原则,经形式化定义与实证验证表明:更强的模态对齐会显著提升从文本到音频的攻击转移效率,揭示能力与安全间的根本矛盾。基于此,我们对近期全模态模型(如 Qwen2.5-Omni、Qwen3-Omni)进行了全面黑盒评估,涵盖三类攻击:文本攻击、文本转移音频攻击、原生音频攻击。结果发现,文本转移音频攻击在仅音频输入条件下表现相当甚至更优,凸显文本漏洞对音频安全的关键影响。进一步实证分析显示,不同攻击方法与模型间均呈现一致趋势:模态对齐越紧密,跨模态攻击转移越有效,有力支持「对齐诅咒」假设。

原文摘要 · Abstract (English)

Recent advances in end-to-end trained omni-models have substantially improved audio capabilities by strengthening text-audio modality alignment. However, whether such alignment inadvertently facilitates the transfer of safety vulnerabilities across modalities remains underexplored. This question is critical as text-based jailbreak attacks are considerably more mature than audio-based ones; if they transfer systematically, current audio safety evaluations may underestimate risks originating from the text modality. In this paper, we introduce the Alignment Curse, a formally characterized and empirically validated principle showing that stronger modality alignment enables more effective transfer of attacks from text to audio, revealing a fundamental tension between capability and safety. Motivated by this principle, we conduct a comprehensive black-box evaluation of three attack categories on recent omni-models (e.g., Qwen2.5-Omni, Qwen3-Omni): text attacks, text-transferred audio attacks, and audio attacks. We find that text-transferred audio attacks perform comparably to, and often better than, audio-based attacks, exhibiting a clear advantage under audio-only access. This suggests that text-based vulnerabilities play a pivotal role in shaping audio safety risks. Finally, we empirically analyze the relationship between modality alignment and transfer effectiveness across attack methods and models, observing consistent support for the Alignment Curse: tighter modality alignment leads to more effective cross-modality attack transfer.

多模态安全对齐诅咒攻击迁移音频安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。