arXiv:2504.17219cs.LGcs.AI2025-04被引 7

用对抗训练提升VAE的生成质量与鲁棒性,打破性能与鲁棒性不可兼得的误区。

Enhancing Variational Autoencoders with Smooth Robust Latent Encoding

  • 通过对抗扰动平滑潜在空间,增强表征泛化能力。
  • 在预训练VAE上后处理,显著提升图像重建和文本引导编辑质量。
  • 适用于需高鲁棒性的生成模型场景,如对抗攻击防御。

变分自编码器(VAEs)在扩散生成模型(如Stable Diffusion)中发挥关键作用,但其鲁棒性问题尚未深入探讨。尽管对抗训练是提升预测模型鲁棒性的成熟方法,却因担心性能与鲁棒性之间的权衡可能导致生成质量下降,而被忽视。本文挑战这一假设,提出Smooth Robust Latent VAE(SRL-VAE),一种新型对抗训练框架,同时提升生成质量和鲁棒性。与传统方法仅关注鲁棒性不同,SRL-VAE通过对抗扰动平滑潜在空间,促进更具泛化性的表示,并结合原始表征正则化以保持生成保真度。作为预训练VAE的后处理步骤,SRL-VAE以极低计算开销提升了图像鲁棒性与保真度。实验表明,该方法在图像重建和文本引导图像编辑任务中均表现更优,且对Nightshade攻击和图像编辑攻击具有更强抵抗力。结果确立了新范式:对抗训练并非生成模型的负担,反而可同步增强保真度与鲁棒性。

原文摘要 · Abstract (English)

Variational Autoencoders (VAEs) have played a key role in scaling up diffusion-based generative models, as in Stable Diffusion, yet questions regarding their robustness remain largely underexplored. Although adversarial training has been an established technique for enhancing robustness in predictive models, it has been overlooked for generative models due to concerns about potential fidelity degradation by the nature of trade-offs between performance and robustness. In this work, we challenge this presumption, introducing Smooth Robust Latent VAE (SRL-VAE), a novel adversarial training framework that boosts both generation quality and robustness. In contrast to conventional adversarial training, which focuses on robustness only, our approach smooths the latent space via adversarial perturbations, promoting more generalizable representations while regularizing with originality representation to sustain original fidelity. Applied as a post-training step on pre-trained VAEs, SRL-VAE improves image robustness and fidelity with minimal computational overhead. Experiments show that SRL-VAE improves both generation quality, in image reconstruction and text-guided image editing, and robustness, against Nightshade attacks and image editing attacks. These results establish a new paradigm, showing that adversarial training, once thought to be detrimental to generative models, can instead enhance both fidelity and robustness.

VAE对抗训练生成模型鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。