arXiv:2604.14591cs.CV2026-04中稿 · the 2026 IEEE/CVF …被引 1

用提示引导图像编辑,精准修改指定区域且保持其余部分不变

Prompt-Guided Image Editing with Masked Logit Nudging in Visual Autoregressive Models

论文配图:Prompt-Guided Image Editing with Masked Logit Nudging in Visual Autoregressive Models
图 1 · 摘自论文原文
  • 通过掩码逻辑牵引,让模型按提示语语义轨迹调整图像
  • 在512px和1024px分辨率下,PIE基准表现最优
  • 速度快于扩散模型,适合需要高效编辑的应用场景

我们解决视觉自回归模型中的提示引导图像编辑问题。给定源图像和目标文本提示,目标是根据提示修改源图像,同时保留与编辑无关的区域。为此,我们提出掩码逻辑牵引(Masked Logit Nudging),利用源图像的标记映射引入引导步骤,使模型在目标提示下的预测与源标记映射对齐。具体地,将固定源编码通过VAR编码转换为逻辑值,沿源-目标提示定义的语义轨迹,将模型预测逻辑值向目标方向牵引。仅在通过专门掩码方案(基于源提示与编辑提示的交叉注意力差异)获得的空间掩码区域内应用编辑。随后引入优化步骤以校正量化误差,提升重建质量。该方法在512px和1024px分辨率下,于PIE基准上达到最佳性能;此外,在COCO(512px)和OpenImages(1024px)上的重建任务中也优于先前方法。总体而言,本方法超越其他视觉自回归模型,性能可媲美甚至超过扩散模型,且速度显著更快。代码已公开于https://github.com/AmirMaEl/MLN。

原文摘要 · Abstract (English)

We address the problem of prompt-guided image editing in visual autoregressive models. Given a source image and a target text prompt, we aim to modify the source image according to the target prompt, while preserving all regions which are unrelated to the requested edit. To this end, we present Masked Logit Nudging, which uses the source image token maps to introduce a guidance step that aligns the model's predictions under the target prompt with these source token maps. Specifically, we convert the fixed source encodings into logits using the VAR encoding, nudging the model's predicted logits towards the targets along a semantic trajectory defined by the source-target prompts. Edits are applied only within spatial masks obtained through a dedicated masking scheme that leverages cross-attention differences between the source and edited prompts. Then, we introduce a refinement to correct quantization errors and improve reconstruction quality. Our approach achieves the best image editing performance on the PIE benchmark at 512px and 1024px resolutions. Beyond editing, our method delivers faithful reconstructions and outperforms previous methods on COCO at 512px and OpenImages at 1024px. Overall, our method outperforms VAR-related approaches and achieves comparable or even better performance than diffusion models, while being much faster. Code is available at 'https://github.com/AmirMaEl/MLN'.

图像编辑自回归模型提示引导

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。