首个基于扩散Transformer的图像编辑框架，实现高分辨率下更优的形状感知编辑。
DiT4Edit: Diffusion Transformer for Image Editing
- 采用DiT结构与统一注意力控制，更好捕捉图像块间长程依赖。
- 使用DPM-Solver反演，编辑步骤减少，生成速度更快。
- 适合需要高精度、任意尺寸图像编辑的研究者和开发者。
尽管基于UNet的图像编辑方法已取得进展，但在高分辨率图像中实现形状感知对象编辑的方法仍不足。相比UNet，扩散Transformer（DiT）能更有效地捕捉图像块间的长程依赖关系，从而生成更高质量的图像。本文提出DiT4Edit，首个基于扩散Transformer的图像编辑框架。具体而言，DiT4Edit采用DPM-Solver反演算法获取逆向隐变量，相比普遍用于UNet框架的DDIM反演算法，显著减少了反演步数。此外，我们设计了统一注意力控制与块合并机制，专为Transformer计算流优化。该集成使框架能在生成更高质量编辑图像的同时加快速度。我们的设计充分利用了DiT的优势，在高分辨率及任意尺寸图像编辑任务中超越了传统UNet结构。大量实验表明，DiT4Edit在多种编辑场景中均表现出色，凸显了扩散Transformer在图像编辑中的潜力。
原文摘要 · Abstract (English)
Despite recent advances in UNet-based image editing, methods for shape-aware object editing in high-resolution images are still lacking. Compared to UNet, Diffusion Transformers (DiT) demonstrate superior capabilities to effectively capture the long-range dependencies among patches, leading to higher-quality image generation. In this paper, we propose DiT4Edit, the first Diffusion Transformer-based image editing framework. Specifically, DiT4Edit uses the DPM-Solver inversion algorithm to obtain the inverted latents, reducing the number of steps compared to the DDIM inversion algorithm commonly used in UNet-based frameworks. Additionally, we design unified attention control and patches merging, tailored for transformer computation streams. This integration allows our framework to generate higher-quality edited images faster. Our design leverages the advantages of DiT, enabling it to surpass UNet structures in image editing, especially in high-resolution and arbitrary-size images. Extensive experiments demonstrate the strong performance of DiT4Edit across various editing scenarios, highlighting the potential of Diffusion Transformers in supporting image editing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。