用自然语言描述编辑意图，实现更精准的图像语义修改。
Describe, Don't Dictate: Semantic Image Editing with Natural Language Intent
- 将指令编辑转为参考图+文本生成，避免重建误差。
- 在Emu Edit上编辑准确率与一致性显著提升。
- 兼容ControlNet等工具，适合需要高精度编辑的场景。
尽管文本到图像生成取得进展，语义图像编辑仍具挑战。基于反演的方法不可避免引入重建误差，而指令式模型则受限于数据集质量与规模。为此，我们提出描述性提示编辑框架DescriptiveEdit。核心思想是将“指令式图像编辑”重构为“参考图像为基础的文本到图像生成”，在不修改架构或反演的前提下，保留预训练文本到图像模型的生成能力。具体地，输入参考图像和提示词，引入跨注意力UNet，新增注意力桥梁将参考图像特征注入到提示到图像的生成过程。由于其文本到图像本质，DescriptiveEdit克服了指令数据集质量局限，可无缝集成ControlNet、IP-Adapter等扩展，且更具可扩展性。在Emu Edit基准测试中，该方法显著提升编辑准确率与一致性。
原文摘要 · Abstract (English)
Despite the progress in text-to-image generation, semantic image editing remains a challenge. Inversion-based algorithms unavoidably introduce reconstruction errors, while instruction-based models mainly suffer from limited dataset quality and scale. To address these problems, we propose a descriptive-prompt-based editing framework, named DescriptiveEdit. The core idea is to re-frame `instruction-based image editing' as `reference-image-based text-to-image generation', which preserves the generative power of well-trained Text-to-Image models without architectural modifications or inversion. Specifically, taking the reference image and a prompt as input, we introduce a Cross-Attentive UNet, which newly adds attention bridges to inject reference image features into the prompt-to-edit-image generation process. Owing to its text-to-image nature, DescriptiveEdit overcomes limitations in instruction dataset quality, integrates seamlessly with ControlNet, IP-Adapter, and other extensions, and is more scalable. Experiments on the Emu Edit benchmark show it improves editing accuracy and consistency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。