用文本嵌入插值实现无需训练的连续图像编辑
The Unreasonable Effectiveness of Text Embedding Interpolation for Continuous Image Steering
- 通过大模型自动生成对比提示对,构建文本编码空间的控制向量
- 在有效范围内缩放向量,实现平滑连续的语义变化
- 方法轻量通用,适用于图文视频生成,适合快速原型开发
我们提出一种无需训练的测试时连续可控图像编辑框架,适用于文本条件生成模型。与依赖额外训练或人工干预的方法不同,我们发现仅在文本嵌入空间进行简单引导即可实现平滑编辑控制。针对目标概念(如提升真实感或改变面部表情),利用大语言模型自动构建少量去偏对比提示对,计算生成器文本编码空间中的引导向量。将该向量直接叠加至输入提示表示,即可沿指定语义轴控制生成。为实现连续控制,提出弹性范围搜索机制,自动识别有效的引导强度区间,避免欠引导(无变化)和过引导(改变其他属性)。在该区间内使用缩放后的同一向量,可获得平滑连续的编辑效果。由于仅修改文本表示,该方法天然适用于各类文本条件模态,包括图像与视频生成。为量化引导连续性,引入新评估指标,衡量不同编辑强度下的语义变化均匀性。实验表明,尽管方法简单轻量,其连续编辑表现仍媲美训练型方案,优于其他无需训练方法。
原文摘要 · Abstract (English)
We present a training-free framework for continuous and controllable image editing at test time for text-conditioned generative models. In contrast to prior approaches that rely on additional training or manual user intervention, we find that a simple steering in the text-embedding space is sufficient to produce smooth edit control. Given a target concept (e.g., enhancing photorealism or changing facial expression), we use a large language model to automatically construct a small set of debiased contrastive prompt pairs, from which we compute a steering vector in the generator's text-encoder space. We then add this vector directly to the input prompt representation to control generation along the desired semantic axis. To obtain a continuous control, we propose an elastic range search procedure that automatically identifies an effective interval of steering magnitudes, avoiding both under-steering (no-edit) and over-steering (changing other attributes). Adding the scaled versions of the same vector within this interval yields smooth and continuous edits. Since our method modifies only textual representations, it naturally generalizes across text-conditioned modalities, including image and video generation. To quantify the steering continuity, we introduce a new evaluation metric that measures the uniformity of semantic change across edit strengths. We compare the continuous editing behavior across methods and find that, despite its simplicity and lightweight design, our approach is comparable to training-based alternatives, outperforming other training-free methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。