提升流模型逆过程精度与语义控制能力
Runge-Kutta Approximation and Decoupled Attention for Rectified Flow Inversion and Semantic Editing
- 用龙格-库塔方法改进流模型逆过程，提高图像重建精度
- 提出解耦注意力机制，使文本与图像注意力分离，增强编辑可控性
- 在图像重构和文本引导编辑任务中表现领先，适合需要高精度生成的场景
最近，修正流（Rectified Flow, RF）模型在生成性能上优于基于DDIM的扩散模型。但在实际应用中仍面临两大挑战：(1) 逆过程精度低，导致与源图像一致性差；(2) 扩散变压器中的多模态注意力纠缠，难以实现精确的注意力控制。为解决第一个问题，我们提出一种基于微分方程龙格-库塔求解器的高效高阶逆过程方法。为应对第二个问题，我们引入解耦扩散变压器注意力（Decoupled Diffusion Transformer Attention, DDTA），该机制将多模态扩散变压器中的文本与图像注意力解耦，实现更精细的语义控制。在图像重建和文本引导编辑任务上的大量实验表明，该方法在保真度和可编辑性方面达到当前最优水平。代码已公开于 https://github.com/wmchen/RKSovler_DDTA。
原文摘要 · Abstract (English)
Rectified flow (RF) models have recently demonstrated superior generative performance compared to DDIM-based diffusion models. However, in real-world applications, they suffer from two major challenges: (1) low inversion accuracy that hinders the consistency with the source image, and (2) entangled multimodal attention in diffusion transformers, which hinders precise attention control. To address the first challenge, we propose an efficient high-order inversion method for rectified flow models based on the Runge-Kutta solver of differential equations. To tackle the second challenge, we introduce Decoupled Diffusion Transformer Attention (DDTA), a novel mechanism that disentangles text and image attention inside the multimodal diffusion transformers, enabling more precise semantic control. Extensive experiments on image reconstruction and text-guided editing tasks demonstrate that our method achieves state-of-the-art performance in terms of fidelity and editability. Code is available at https://github.com/wmchen/RKSovler_DDTA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。