arXiv:2410.02240cs.CVcs.AI2024-10被引 3

提出新方法生成语义一致的对抗样本,更快更逼真。

SCA: Improve Semantic Consistent in Unrestricted Adversarial Attacks via DDPM Inversion

  • 用扩散模型反演提取可编辑噪声图,结合多模态大模型引导语义。
  • 生成对抗样本平均快12倍,且语义变化几乎不可察觉。
  • 适合研究对抗攻击鲁棒性或高效生成的从业者参考。

基于深度神经网络的系统易受对抗攻击。无限制对抗攻击通常通过修改图像的语义内容(如颜色或纹理)生成既有效又逼真的对抗样本。近期工作利用扩散模型反演将图像映射到隐空间,在其中通过引入扰动操纵高层语义。然而,这些方法常导致去噪输出中出现显著语义失真,且效率低下。本文提出一种新框架Semantic-Consistent Unrestricted Adversarial Attacks (SCA),采用反演方法提取可编辑噪声图,并利用多模态大语言模型(MLLM)在全过程提供语义指导。在MLLM提供的丰富语义信息下,使用一系列可编辑噪声图进行DDPM去噪,并结合DPM Solver++加速采样过程,实现高效且语义一致的生成。相比现有方法,该框架能高效生成语义变化极小的对抗样本,首次引入语义一致对抗样本(SCAE)。大量实验与可视化表明,SCA具有极高效率,尤其在平均速度上比当前最优攻击快12倍。代码已公开于https://github.com/Pan-Zihao/SCA。

原文摘要 · Abstract (English)

Systems based on deep neural networks are vulnerable to adversarial attacks. Unrestricted adversarial attacks typically manipulate the semantic content of an image (e.g., color or texture) to create adversarial examples that are both effective and photorealistic. Recent works have utilized the diffusion inversion process to map images into a latent space, where high-level semantics are manipulated by introducing perturbations. However, they often result in substantial semantic distortions in the denoised output and suffer from low efficiency. In this study, we propose a novel framework called Semantic-Consistent Unrestricted Adversarial Attacks (SCA), which employs an inversion method to extract edit-friendly noise maps and utilizes a Multimodal Large Language Model (MLLM) to provide semantic guidance throughout the process. Under the condition of rich semantic information provided by MLLM, we perform the DDPM denoising process of each step using a series of edit-friendly noise maps and leverage DPM Solver++ to accelerate this process, enabling efficient sampling with semantic consistency. Compared to existing methods, our framework enables the efficient generation of adversarial examples that exhibit minimal discernible semantic changes. Consequently, we for the first time introduce Semantic-Consistent Adversarial Examples (SCAE). Extensive experiments and visualizations have demonstrated the high efficiency of SCA, particularly in being on average 12 times faster than the state-of-the-art attacks. Our code can be found at https://github.com/Pan-Zihao/SCA.

对抗攻击扩散模型语义一致性高效生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。