用语义退化条件替代空提示,提升图像生成的细节控制精度
Guiding Diffusion Models with Semantically Degraded Conditions
- 用退化的条件向量替代传统空提示,实现更精细的引导机制
- 在多个模型上显著提升组合任务准确率和图文对齐度
- 无需额外训练,轻量级插件式模块,适合各类扩散模型
Classifier-Free Guidance(CFG)是现代文生图模型的核心,但其依赖语义空白的空提示($oldsymbol{ ext{null}}$),导致引导信号易产生几何纠缠,限制了生成精度,尤其在复杂组合任务中表现不佳。本文提出条件退化引导(CDG),用有策略退化的条件向量$oldsymbol{c}_{ ext{deg}}$替代空提示。该方法将引导从粗粒度的“好与无”对比,转变为更精细的“好与近好”区分,促使模型捕捉细粒度语义差异。我们发现,Transformer文本编码器中的词元分为两类:内容词元编码物体语义,上下文聚合词元捕捉全局信息。通过仅退化前者构建$oldsymbol{c}_{ ext{deg}}$,无需外部模型或训练。在Stable Diffusion 3、FLUX和Qwen-Image等多架构上验证,CDG显著提升组合准确性与图文对齐效果。作为轻量级、即插即用模块,计算开销极小。本工作挑战了对静态、信息稀疏负样本的依赖,确立新原则:构建自适应、语义感知的负样本是实现精确语义控制的关键。代码已开源。
原文摘要 · Abstract (English)
Classifier-Free Guidance (CFG) is a cornerstone of modern text-to-image models, yet its reliance on a semantically vacuous null prompt ($\varnothing$) generates a guidance signal prone to geometric entanglement. This is a key factor limiting its precision, leading to well-documented failures in complex compositional tasks. We propose Condition-Degradation Guidance (CDG), a novel paradigm that replaces the null prompt with a strategically degraded condition, $\boldsymbol{c}_{\text{deg}}$. This reframes guidance from a coarse "good vs. null" contrast to a more refined "good vs. almost good" discrimination, thereby compelling the model to capture fine-grained semantic distinctions. We find that tokens in transformer text encoders split into two functional roles: content tokens encoding object semantics, and context-aggregating tokens capturing global context. By selectively degrading only the former, CDG constructs $\boldsymbol{c}_{\text{deg}}$ without external models or training. Validated across diverse architectures including Stable Diffusion 3, FLUX, and Qwen-Image, CDG markedly improves compositional accuracy and text-image alignment. As a lightweight, plug-and-play module, it achieves this with negligible computational overhead. Our work challenges the reliance on static, information-sparse negative samples and establishes a new principle for diffusion guidance: the construction of adaptive, semantically-aware negative samples is critical to achieving precise semantic control. Code is available at https://github.com/Ming-321/Classifier-Degradation-Guidance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。