无需训练即可精准编辑图像,通过智能路由选择最佳注意力操作。
AttnRouter: Per-Category Attention Routing for Training-Free Image Editing on MMDiT

- 用局部注入法在单次前向中融合源图与噪声的键值对,实现高效编辑。
- 按类别路由不同编辑策略,使复合得分提升6.4%,接近人工标注效果。
- 发现早期去噪步骤(S0-7)和中间层(L0-15)最有效,α取值0.3~0.5为佳。
我们研究了在Qwen-Image-Edit-2511(一个60层的多模态扩散变换器)上进行无训练图像编辑。提出KVInject方法,在局部层/步带内将源图一半的键值投影α混合注入噪声一半,比传统两步法更简单且避免提示不匹配问题(复合得分下降31%)。发现不同编辑类型无通用最优操作,因此设计AttnRouter:按类别路由至最适操作,使用真实类别时复合得分提升6.4%;即使使用CLIP零样本分类器(仅55%准确率),仍弥补98%差距。通过层、步、α消融实验定位有效子电路:早期去噪步骤(S0-7)注入可恢复近全部增益,而过早或过晚层注入无效;α在[0.3, 0.5]区间表现稳定。也报告负结果:简单键值缩放无法超越基线,激进变体导致生成崩溃(复合得分0.084)。代码、预计算路由表及100样本分层数据集已公开。
原文摘要 · Abstract (English)
We study training-free image editing on Qwen-Image-Edit-2511, a 60-block multi-modal diffusion transformer (MMDiT) that concatenates noise and source-image tokens within a single attention stream. We make three contributions. (i) We introduce KVInject, a single-forward attention manipulation that alpha-blends source-half key/value projections into the noise-half within a localized layer/step band. KVInject is simpler than the classical two-pass MasaCtrl recipe and avoids the prompt-mismatch failure mode that disables MasaCtrl on MMDiT (composite score drops 31% versus baseline). (ii) We show that no single attention operation dominates across edit types, motivating AttnRouter, a per-category routing table that dispatches edits to the operation that best preserves source structure for that type. With ground-truth categories the router improves the CLIP-T+DINO-I composite by 6.4% over the editing baseline; an automatic CLIP zero-shot classifier closes 98% of this gap despite only 55% category accuracy. (iii) Through layer-, step-, and alpha-band ablations we localize the editing-effective attention sub-circuit: K/V injection in early denoising steps (S0-7) recovers nearly all of the gain of full-step injection, while injection in early (L0-15) or late (L45-60) layer bands fails to drive editing entirely; alpha in [0.3, 0.5] is a stable sweet spot. We also report negative results that highlight what does not transfer from the UNet folklore: simple K/V rescaling never beats baseline and aggressive variants collapse generation entirely (composite 0.084). We release code, pre-computed routing tables, and a 100-sample stratified subset of ImgEdit-Bench used in all ablations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。