无需文字提示,仅用一张图片即可实现精准图像编辑。
Language-Free Generative Editing from One Visual Example
- 通过视觉示例直接学习编辑条件,摆脱语言依赖。
- 在雨天、模糊等常见效果上表现优于现有方法。
- 无需微调,适合快速实用的图像修改场景。
文本引导的扩散模型虽能实现直观的图像编辑,但我们在实验中发现,当前最先进方法在雨天、模糊等日常变换上表现不佳。这源于训练中语言监督弱且不一致,导致语义与视觉对齐差。现有方案多依赖额外微调或更强文本条件,但计算成本高。我们提出视觉扩散条件(VDC),一种无需训练的框架,从成对图像(有/无目标效果)中直接提取视觉条件,通过新型条件引导机制实现精准语言无关编辑。配套的反演修正步骤缓解了DDIM反演中的重建误差,保持细节真实感。在多种任务中,VDC均超越训练自由与全微调的文本编辑方法。代码与模型已开源。
原文摘要 · Abstract (English)
Text-guided diffusion models have advanced image editing by enabling intuitive control through language. However, despite their strong capabilities, we surprisingly find that SOTA methods struggle with simple, everyday transformations such as rain or blur. We attribute this limitation to weak and inconsistent textual supervision during training, which leads to poor alignment between language and vision. Existing solutions often rely on extra finetuning or stronger text conditioning, but suffer from high data and computational requirements. We argue that diffusion-based editing capabilities aren't lost but merely hidden from text. The door to cost-efficient visual editing remains open, and the key lies in a vision-centric paradigm that perceives and reasons about visual change as humans do, beyond words. Inspired by this, we introduce Visual Diffusion Conditioning (VDC), a training-free framework that learns conditioning signals directly from visual examples for precise, language-free image editing. Given a paired example -one image with and one without the target effect- VDC derives a visual condition that captures the transformation and steers generation through a novel condition-steering mechanism. An accompanying inversion-correction step mitigates reconstruction errors during DDIM inversion, preserving fine detail and realism. Across diverse tasks, VDC outperforms both training-free and fully fine-tuned text-based editing methods. The code and models are open-sourced at https://omaralezaby.github.io/vdc/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。