无需训练即可精准控制扩散模型编辑强度,提升局部修改效果。
Dual-Channel Attention Guidance for Training-Free Image Editing Control in Diffusion Transformers
- 同时调控注意力的键值通道,实现粗细结合的编辑控制
- 在700张图像上实现4.9%的局部删除改进,3.2%的对象添加优化
- 适合需要精细调节编辑强度的研究者和开发者
基于扩散Transformer(DiT)架构的图像编辑模型,亟需无需训练的编辑强度控制。现有方法仅操纵注意力中的键(Key)空间,忽略对特征聚合起关键作用的值(Value)空间。本文首次揭示,DiT多模态注意力层中键与值投影均呈现显著的偏置-增量结构,即标记嵌入紧密聚集于层特定的偏置向量周围。基于此,提出无需训练的双通道注意力引导(DCAG)框架,同步调控键通道(决定关注位置)与值通道(决定聚合内容)。理论分析表明,键通道通过非线性Softmax函数起作用,作为粗调旋钮;值通道通过线性加权求和实现细调补充。两者构成二维参数空间(δ_k, δ_v),可实现比单通道方法更精确的编辑保真度权衡。在PIE-Bench基准测试(700张图像,10类编辑任务)中,DCAG在所有保真度指标上均优于仅调控键的方法,尤其在局部编辑任务中表现突出:对象删除任务降低4.9% LPIPS,对象添加任务降低3.2% LPIPS。
原文摘要 · Abstract (English)
Training-free control over editing intensity is a critical requirement for diffusion-based image editing models built on the Diffusion Transformer (DiT) architecture. Existing attention manipulation methods focus exclusively on the Key space to modulate attention routing, leaving the Value space -- which governs feature aggregation -- entirely unexploited. In this paper, we first reveal that both Key and Value projections in DiT's multi-modal attention layers exhibit a pronounced bias-delta structure, where token embeddings cluster tightly around a layer-specific bias vector. Building on this observation, we propose Dual-Channel Attention Guidance (DCAG), a training-free framework that simultaneously manipulates both the Key channel (controlling where to attend) and the Value channel (controlling what to aggregate). We provide a theoretical analysis showing that the Key channel operates through the nonlinear softmax function, acting as a coarse control knob, while the Value channel operates through linear weighted summation, serving as a fine-grained complement. Together, the two-dimensional parameter space $(δ_k, δ_v)$ enables more precise editing-fidelity trade-offs than any single-channel method. Extensive experiments on the PIE-Bench benchmark (700 images, 10 editing categories) demonstrate that DCAG consistently outperforms Key-only guidance across all fidelity metrics, with the most significant improvements observed in localized editing tasks such as object deletion (4.9% LPIPS reduction) and object addition (3.2% LPIPS reduction).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。