用少量数据实现精准图像编辑，效率远超传统方法
In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer
- 不改结构，直接利用大模型上下文生成能力进行编辑
- 仅需0.1%数据和1%可训练参数，达到顶尖效果
- 适合追求高效高精度图像编辑的研究者与开发者
基于指令的图像编辑可通过自然语言提示实现精确修改，但现有方法存在精度与效率的权衡：微调需超1000万样本及大量算力，而零训练方法则指令理解能力弱。本文提出ICEdit，通过三项创新解决该问题：(1) 无需架构修改的上下文编辑范式；(2) 极少参数的高效微调以提升质量；(3) 早期滤波推理时缩放，利用视觉语言模型筛选高质量噪声样本以提升效率。实验表明，ICEdit仅需先前方法0.1%的训练数据和1%的可训练参数，即达领先性能。本方法为指令式图像编辑中的精度与效率平衡提供了新范式。代码与演示见https://river-zhang.github.io/ICEdit-gh-pages/
原文摘要 · Abstract (English)
Instruction-based image editing enables precise modifications via natural language prompts, but existing methods face a precision-efficiency tradeoff: fine-tuning demands massive datasets (>10M) and computational resources, while training-free approaches suffer from weak instruction comprehension. We address this by proposing ICEdit, which leverages the inherent comprehension and generation abilities of large-scale Diffusion Transformers (DiTs) through three key innovations: (1) An in-context editing paradigm without architectural modifications; (2) Minimal parameter-efficient fine-tuning for quality improvement; (3) Early Filter Inference-Time Scaling, which uses VLMs to select high-quality noise samples for efficiency. Experiments show that ICEdit achieves state-of-the-art editing performance with only 0.1\% of the training data and 1\% trainable parameters compared to previous methods. Our approach establishes a new paradigm for balancing precision and efficiency in instructional image editing. Codes and demos can be found in https://river-zhang.github.io/ICEdit-gh-pages/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。