提出联合优化音视频的越狱攻击，显著提升对语音模型的安全威胁
On Optimizing Multimodal Jailbreaks for Spoken Language Models
- 设计双模态优化框架JAMA，同步扰动文本与音频
- 在4个SLS上实现1.5到20倍的越狱成功率提升
- 适合安全研究者关注多模态模型的防御漏洞
随着语音语言模型（SLMs）融合语音与文本模态，它们继承了其大型语言模型（LLM）的安全部署缺陷，并引入了更广的攻击面。已有研究显示SLMs易受越狱攻击，即通过对抗性提示诱导有害响应。然而现有攻击大多局限于单模态，仅独立优化文本或音频。本文提出基于梯度的多模态越狱方法——JAMA（联合音视频多模态攻击），结合贪婪坐标梯度（GCG）用于文本和投影梯度下降（PGD）用于音频，实现双模态同时扰动。在四个先进SLMs及四种音频类型上的评估表明，JAMA的越狱成功率比单模态攻击高出1.5至20倍。我们分析了联合攻击的运行机制，发现序列近似方法使效率提升4至6倍。结果表明，单模态安全性不足以保障鲁棒的SLMs。代码与数据已公开于https://repos.lsv.uni-saarland.de/akrishnan/multimodal-jailbreak-slm。
原文摘要 · Abstract (English)
As Spoken Language Models (SLMs) integrate speech and text modalities, they inherit the safety vulnerabilities of their LLM backbone while introducing an expanded attack surface. SLMs have been previously shown to be susceptible to jailbreaking, where adversarial prompts induce harmful responses. Yet existing attacks largely remain unimodal, optimizing either text or audio in isolation. We explore gradient-based multimodal jailbreaks by introducing JAMA (Joint Audio-text Multimodal Attack), a joint multimodal optimization framework combining Greedy Coordinate Gradient (GCG) for text and Projected Gradient Descent (PGD) for audio, to simultaneously perturb both modalities. Evaluations across four state-of-the-art SLMs and four audio types demonstrate that JAMA surpasses unimodal jailbreak rate by 1.5x to 20x. We analyze the operational dynamics of this joint attack and show that a sequential approximation method makes it 4x to 6x faster. Our findings suggest that unimodal safety is insufficient for robust SLMs. The code and data are available at https://repos.lsv.uni-saarland.de/akrishnan/multimodal-jailbreak-slm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。