用注意力蒸馏让生成图像精准继承参考图的风格与细节
Attention Distillation: A Unified Approach to Visual Characteristics Transfer
- 通过优化潜在空间中的注意力特征实现风格迁移
- 在多个数据集上显著提升风格、外观和纹理的转移效果
- 适合需要高质量图像风格迁移的研究与应用开发者
生成式扩散模型展现出对图像风格与语义的内在理解。本文利用预训练扩散网络的自注意力特征，将参考图像的视觉特征迁移到生成图像中。不同于以往将这些特征作为即插即用属性的方法，我们提出一种基于理想与当前风格化结果间注意力差异的新型蒸馏损失，并在潜在空间中通过反向传播优化合成图像。进一步提出改进的分类器引导方法，将注意力蒸馏损失融入去噪采样过程，加速生成并拓展应用场景。大量实验表明，该方法在风格、外观和纹理迁移方面表现卓越。代码已开源：https://github.com/xugao97/AttentionDistillation。
原文摘要 · Abstract (English)
Recent advances in generative diffusion models have shown a notable inherent understanding of image style and semantics. In this paper, we leverage the self-attention features from pretrained diffusion networks to transfer the visual characteristics from a reference to generated images. Unlike previous work that uses these features as plug-and-play attributes, we propose a novel attention distillation loss calculated between the ideal and current stylization results, based on which we optimize the synthesized image via backpropagation in latent space. Next, we propose an improved Classifier Guidance that integrates attention distillation loss into the denoising sampling process, further accelerating the synthesis and enabling a broad range of image generation applications. Extensive experiments have demonstrated the extraordinary performance of our approach in transferring the examples' style, appearance, and texture to new images in synthesis. Code is available at https://github.com/xugao97/AttentionDistillation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。