arXiv:2412.02687cs.CV2024-12ICCV被引 8

让单步文生图模型更可控,用负提示消除不想要的元素

Supercharged One-step Text-to-Image Diffusion Models with Negative Prompts

  • 在中间表示空间利用交叉注意力抑制不需要的视觉特征
  • 生成质量提升,HPSv2得分达31.21,创单步模型新纪录
  • 兼容现有训练框架,仅增加1.89%计算量,适合高效图像生成场景

实时图像生成需求推动了单步扩散模型的发展,其生成速度远超传统多步方法。然而,效率提升常以控制能力下降为代价。尽管负提示在多步模型中通过无分类器引导(CFG)实现精细控制,但直接应用于单步模型会因缺乏迭代优化而产生融合伪影,降低输出质量。为此,我们提出负向规避导向注意力(NASA),通过跨注意力机制在中间表征空间抑制不希望的视觉属性,避免输出空间引导带来的融合问题。该方法计算开销极低,仅比原模型增加1.89%的浮点运算量,远低于CFG的计算翻倍。NASA可无缝集成到现有时间步蒸馏框架中,显著提升学生模型输出质量。实验表明,NASA使单步扩散模型在控制性和生成质量上实现大幅提升,达到31.21的HPSv2分数,刷新当前最优水平。

原文摘要 · Abstract (English)

The escalating demand for real-time image synthesis has driven significant advancements in one-step diffusion models, which inherently offer expedited generation speeds compared to traditional multi-step methods. However, this enhanced efficiency is frequently accompanied by a compromise in the controllability of image attributes. While negative prompting, typically implemented via classifier-free guidance (CFG), has proven effective for fine-grained control in multi-step models, its application to one-step generators remains largely unaddressed. Due to the lack of iterative refinement, as in multi-step diffusion, directly applying CFG to one-step generation leads to blending artifacts and diminished output quality. To fill this gap, we introduce \textbf{N}egative-\textbf{A}way \textbf{S}teer \textbf{A}ttention (NASA), an efficient method that integrates negative prompts into one-step diffusion models. NASA operates within the intermediate representation space by leveraging cross-attention mechanisms to suppress undesired visual attributes. This strategy avoids the blending artifacts inherent in output-space guidance and achieves high efficiency, incurring only a minimal 1.89\% increase in FLOPs compared to the computational doubling of CFG. Furthermore, NASA can be seamlessly integrated into existing timestep distillation frameworks, enhancing the student's output quality. Experimental results demonstrate that NASA substantially improves controllability and output quality, achieving an HPSv2 score of \textbf{31.21}, setting a new state-of-the-art benchmark for one-step diffusion models.

文生图扩散模型负提示单步生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。