攻击音频模型编码器，用通用扰动骗其生成指定文本输出。
Breaking Audio Large Language Models by Attacking Only the Encoder: A Universal Targeted Latent-Space Audio Attack
- 在音频潜在空间学习通用扰动，跨输入和说话人有效
- 对Qwen2-Audio-7B-Instruct攻击成功率高，听觉感知失真小
- 无需语言模型信息，揭示编码器级新安全漏洞
音频-语言模型通过结合音频编码器与大语言模型实现多模态推理，但也引入了新的安全风险。本文提出一种通用目标性潜在空间攻击，即在编码器层级进行对抗攻击，通过操纵音频潜在表示，诱导下游语言生成产生攻击者指定的输出。与以往波形级或输入特定的攻击不同，该方法学习一个可泛化到多种输入和说话人的通用扰动，且无需访问语言模型。在Qwen2-Audio-7B-Instruct上的实验表明，该攻击在保持极低听觉感知失真条件下实现了稳定的高成功率，揭示了多模态系统中编码器层级存在一个此前被严重忽视的关键攻击面。
原文摘要 · Abstract (English)
Audio-language models combine audio encoders with large language models to enable multimodal reasoning, but they also introduce new security vulnerabilities. We propose a universal targeted latent space attack, an encoder-level adversarial attack that manipulates audio latent representations to induce attacker-specified outputs in downstream language generation. Unlike prior waveform-level or input-specific attacks, our approach learns a universal perturbation that generalizes across inputs and speakers and does not require access to the language model. Experiments on Qwen2-Audio-7B-Instruct demonstrate consistently high attack success rates with minimal perceptual distortion, revealing a critical and previously underexplored attack surface at the encoder level of multimodal systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。