无需训练即可多概念编辑,保留图像细节与身份特征
Training-Free Multi-Concept Image Editing
- 利用预训练LoRA适配器的稳定特征进行动态加权编辑
- 在InstructPix2Pix和ComposLoRA上超越现有方法,保持身份一致
- 适用于无参考样本的多概念组合,避免概念冲突
基于扩散模型的无训练图像编辑虽理想但复杂,现有优化方法在零样本编辑中仍难以保留身份和细节(如面部结构、表面纹理或物体几何)。为此,我们提出概念蒸馏采样(CDS),首次构建统一的无训练、无目标多概念编辑框架。CDS通过预训练LoRA适配器的确定性特征锚定编辑过程,结合有序时间步、正则化与负提示引导的稳定蒸馏主干,以及新颖的动态加权机制,实现多视觉概念在扩散过程中直接可控组合,利用空间感知先验避免概念冲突。该方法无需目标参考样本即可保持实例概念身份。大量定量与定性评估表明,CDS在InstructPix2Pix和ComposLoRA基准上达到新最优性能。
原文摘要 · Abstract (English)
Training-free image editing with diffusion models is highly desirable yet is complex and remains a significant challenge. While recent optimisation-based methods achieve strong zero-shot edits from text, they still struggle to preserve identity and capture intricate details, such as facial structure, surface texture, or object-specific geometry, that exist below the level of linguistic abstraction. To address this fundamental gap, we propose Concept Distillation Sampling (CDS). To the best of our knowledge, we are the first to introduce a unified, training-free framework for target-less, multi-concept image editing. CDS overcomes this linguistic bottleneck of previous methods by anchoring the editing process in the certainty of pretrained LoRA adapters. We integrate a highly stable distillation backbone (featuring ordered timesteps, regularisation, and negative-prompt guidance) with a novel dynamic weighting mechanism. This approach enables the composition and control of multiple visual concepts directly within the diffusion process, utilising spatially-aware priors from pretrained LoRA adapters without causing concept clashing. Our method preserves instance concept identity without requiring reference samples of the desired edit. Extensive quantitative and qualitative evaluations demonstrate that CDS establishes a new state-of-the-art over existing training-free editing and multi-LoRA composition methods on the InstructPix2Pix and ComposLoRA benchmarks. Project Page: https://nickyfot.github.io/cds/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。