让多个指令并行编辑图像，避免错误累积和修改不全
Disentangling Instruction Influence in Diffusion Transformers for Parallel Multi-Instruction-Guided Image Editing
- 通过分析注意力机制，分离每个指令的影响范围
- 在单次去噪中完成多指令编辑，减少步骤且提升准确率
- 适合需要快速精准批量修改图像的用户
指令引导的图像编辑允许用户用自然语言指定修改，提升灵活性。现有框架中，扩散Transformer（DiTs）在可扩展性和性能上优于基于U-Net的扩散模型。然而，在真实场景中常需同时执行多个指令，逐步编辑会因误差累积导致质量下降，而将多指令合并到单一提示中则常因指令冲突导致修改不完整。本文提出指令影响解耦（IID）框架，支持在单个去噪过程中并行执行多个指令，专为DiT模型设计。通过分析DiTs中的自注意力机制，我们识别出多指令设置下的独特注意力模式，并推导出针对每条指令的注意力掩码，以解耦各指令的影响。这些掩码指导编辑过程，确保修改局部化且非编辑区域保持一致。在开源与自建数据集上的大量实验表明，与现有基线相比，IID在减少扩散步数的同时提升了保真度和指令完成率。代码将在论文录用后公开。
原文摘要 · Abstract (English)
Instruction-guided image editing enables users to specify modifications using natural language, offering more flexibility and control. Among existing frameworks, Diffusion Transformers (DiTs) outperform U-Net-based diffusion models in scalability and performance. However, while real-world scenarios often require concurrent execution of multiple instructions, step-by-step editing suffers from accumulated errors and degraded quality, and integrating multiple instructions with a single prompt usually results in incomplete edits due to instruction conflicts. We propose Instruction Influence Disentanglement (IID), a novel framework enabling parallel execution of multiple instructions in a single denoising process, designed for DiT-based models. By analyzing self-attention mechanisms in DiTs, we identify distinctive attention patterns in multi-instruction settings and derive instruction-specific attention masks to disentangle each instruction's influence. These masks guide the editing process to ensure localized modifications while preserving consistency in non-edited regions. Extensive experiments on open-source and custom datasets demonstrate that IID reduces diffusion steps while improving fidelity and instruction completion compared to existing baselines. The codes will be publicly released upon the acceptance of the paper.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。