通过识别DiT关键层实现无需训练的稳定图像编辑
Stable Flow: Vital Layers for Training-Free Image Editing
- 定位DiT中影响图像生成的关键层，实现精准特征注入
- 同一机制支持非刚性变形与物体添加，编辑效果稳定一致
- 改进图像逆向重建方法，适用于真实图像编辑
扩散模型已彻底改变内容生成与编辑领域。近期模型采用扩散Transformer（DiT）替代传统UNet架构，并使用流匹配提升训练与采样效率，但生成多样性有限。本文利用这一局限性，通过选择性注入注意力特征实现一致的图像编辑。主要挑战在于，与UNet基于粗到细的结构不同，DiT缺乏明确的层级结构，难以确定注入位置。为此，我们提出自动识别DiT中对图像形成至关重要的“关键层”的方法，并展示这些层如何通过统一机制实现多种可控稳定编辑，包括非刚性修改与物体添加。为进一步支持真实图像编辑，我们引入改进的图像逆向重建方法以适配流模型。最后，通过定性、定量对比及用户研究评估方法有效性，验证其在多个应用场景中的表现。项目页面见https://omriavrahami.com/stable-flow。
原文摘要 · Abstract (English)
Diffusion models have revolutionized the field of content synthesis and editing. Recent models have replaced the traditional UNet architecture with the Diffusion Transformer (DiT), and employed flow-matching for improved training and sampling. However, they exhibit limited generation diversity. In this work, we leverage this limitation to perform consistent image edits via selective injection of attention features. The main challenge is that, unlike the UNet-based models, DiT lacks a coarse-to-fine synthesis structure, making it unclear in which layers to perform the injection. Therefore, we propose an automatic method to identify "vital layers" within DiT, crucial for image formation, and demonstrate how these layers facilitate a range of controlled stable edits, from non-rigid modifications to object addition, using the same mechanism. Next, to enable real-image editing, we introduce an improved image inversion method for flow models. Finally, we evaluate our approach through qualitative and quantitative comparisons, along with a user study, and demonstrate its effectiveness across multiple applications. The project page is available at https://omriavrahami.com/stable-flow
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。