黑客可用电音干扰让语音大模型误响应或崩溃
Attacker's Noise Can Manipulate Your Audio-based LLM in the Real World
- 用隐蔽音频扰动欺骗语音大模型执行特定指令
- 播放干扰噪声使模型响应质量显著下降
- 攻击可跨设备传播，影响无辜用户
本文研究了语音大语言模型（ALLMs）如Qwen2-Audio在真实场景下的漏洞。实验表明，攻击者可生成隐蔽的音频扰动，诱使模型对唤醒词（如“Hey Qwen”）做出响应，或触发有害行为（如“更改我的日程事件”）。此外，在用户交互过程中播放对抗性背景噪声，会显著降低模型响应质量。关键的是，此类攻击可在真实环境中规模化实施，通过空气传播影响其他无关用户。研究还探讨了攻击的可迁移性及潜在防御措施。
原文摘要 · Abstract (English)
This paper investigates the real-world vulnerabilities of audio-based large language models (ALLMs), such as Qwen2-Audio. We first demonstrate that an adversary can craft stealthy audio perturbations to manipulate ALLMs into exhibiting specific targeted behaviors, such as eliciting responses to wake-keywords (e.g., "Hey Qwen"), or triggering harmful behaviors (e.g. "Change my calendar event"). Subsequently, we show that playing adversarial background noise during user interaction with the ALLMs can significantly degrade the response quality. Crucially, our research illustrates the scalability of these attacks to real-world scenarios, impacting other innocent users when these adversarial noises are played through the air. Further, we discuss the transferrability of the attack, and potential defensive measures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。