用示例对实现高效图像编辑，速度提升4倍且效果更优。
Towards Efficient Exemplar Based Image Editing with Multimodal VLMs
- 通过文本-图像扩散模型与多模态VLM，无需优化直接完成示例编辑。
- 在多种编辑类型上优于基线方法，速度比现有方案快约4倍。
- 适合需要快速、准确图像修改的设计师或AI应用开发者。
文本到图像扩散模型已推动众多图像编辑应用的发展。然而，仅靠文本描述难以有效捕捉所有类型的图像编辑，某些编辑的模糊性更适合通过示例对（即编辑前后的图像对）表达。本文研究示例驱动的图像编辑任务——将示例对中的编辑效果迁移至内容图像。我们提出一个无需优化的端到端框架，利用预训练的文本到图像扩散模型与多模态视觉语言模型（VLMs）。实验表明，该方法在多种编辑类型上均优于基线，且推理速度提升约4倍。
原文摘要 · Abstract (English)
Text-to-Image Diffusion models have enabled a wide array of image editing applications. However, capturing all types of edits through text alone can be challenging and cumbersome. The ambiguous nature of certain image edits is better expressed through an exemplar pair, i.e., a pair of images depicting an image before and after an edit respectively. In this work, we tackle exemplar-based image editing -- the task of transferring an edit from an exemplar pair to a content image(s), by leveraging pretrained text-to-image diffusion models and multimodal VLMs. Even though our end-to-end pipeline is optimization-free, our experiments demonstrate that it still outperforms baselines on multiple types of edits while being ~4x faster.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。