用生成模型隐空间优化对抗样本,提升跨模型迁移性与鲁棒性
Latent Transfer Attack: Adversarial Examples via Generative Latent Spaces
- 在Stable Diffusion的隐空间中优化扰动,而非像素空间
- 对多种模型攻击成功率超基准方法,且抗缩放裁剪等预处理
- 适合研究模型鲁棒性或生成式对抗攻击的学者
对抗攻击是评估现代视觉模型鲁棒性的核心工具,但多数方法在像素空间中以ℓ∞或ℓ₂约束直接优化扰动。这类方法虽在白盒场景有效,却常产生高频、纹理类噪声,易受缩放、裁剪等预处理破坏,且跨架构迁移能力差。本文提出LTA(Latent Transfer Attack),在预训练Stable Diffusion VAE的隐空间中优化扰动。给定一张干净图像,先将其编码为隐向量,再优化该隐表示以最大化代理分类器损失,解码后软性施加像素空间ℓ∞预算。为增强对分辨率不匹配和标准输入流程的鲁棒性,引入随机缩放、插值和裁剪的期望变换(EOT),并周期性应用隐空间高斯平滑以抑制伪影、稳定优化。在多类CNN与视觉Transformer模型上,LTA展现出强迁移攻击效果,生成的空间连贯、以低频为主的扰动,与像素空间基线有本质差异,位于迁移性与质量权衡的更优位置。结果表明,预训练生成隐空间是对抗优化的有效且结构化领域,连接了鲁棒性评估与现代生成先验。
原文摘要 · Abstract (English)
Adversarial attacks are a central tool for probing the robustness of modern vision models, yet most methods optimize perturbations directly in pixel space under $\ell_\infty$ or $\ell_2$ constraints. While effective in white-box settings, pixel-space optimization often produces high-frequency, texture-like noise that is brittle to common preprocessing (e.g., resizing and cropping) and transfers poorly across architectures. We propose $\textbf{LTA}$ ($\textbf{L}$atent $\textbf{T}$ransfer $\textbf{A}$ttack), a transfer-based attack that instead optimizes perturbations in the latent space of a pretrained Stable Diffusion VAE. Given a clean image, we encode it into a latent code and optimize the latent representation to maximize a surrogate classifier loss, while softly enforcing a pixel-space $\ell_\infty$ budget after decoding. To improve robustness to resolution mismatch and standard input pipelines, we incorporate Expectation Over Transformations (EOT) via randomized resizing, interpolation, and cropping, and apply periodic latent Gaussian smoothing to suppress emerging artifacts and stabilize optimization. Across a suite of CNN and vision-transformer targets, LTA achieves strong transfer attack success while producing spatially coherent, predominantly low-frequency perturbations that differ qualitatively from pixel-space baselines and occupy a distinct point in the transfer-quality trade-off. Our results highlight pretrained generative latent spaces as an effective and structured domain for adversarial optimization, bridging robustness evaluation with modern generative priors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。