arXiv:2512.03996cs.CVcs.AI2025-12被引 1

通过扰动文本嵌入提升图像生成质量,无需额外计算开销。

Highly Efficient Test-Time Scaling for T2I Diffusion Models with Text Embedding Perturbation

  • 引入分步文本嵌入扰动,结合频域引导的噪声调度。
  • 在多个基准上显著提升生成质量,且几乎不增加计算量。
  • 适合希望优化扩散模型生成效果的研究者与开发者。

测试时扩展(TTS)通过增加随机采样并基于规则和指标评估样本以获得更好结果。然而,在文本到图像(T2I)扩散模型中,现有工作多关注搜索策略和奖励模型,而未探讨扩散模型中噪声的随机性对方法性能的影响。本文分析了T2I模型中的随机性,并提出一种新的随机形式:文本嵌入扰动,该方法与原有的SDE注入噪声协同作用,增强生成多样性与质量。我们从频域角度分析不同随机形式的影响,发现空间噪声偏好低频成分(早期步骤),而文本嵌入扰动增强高频细节(后期步骤),二者在频域上具有互补性,可弥补空间噪声在高频操控上的局限。同时,文本嵌入在不同生成阶段对扰动的容忍度不同。本方法包含两项核心设计:(1) 引入分步文本嵌入扰动,结合频域引导的噪声调度与空间噪声扰动;(2) 根据扰动在不同频段的生成贡献及容忍度,自适应调整扰动强度。该方法可无缝集成至现有TTS框架,在多个基准上实现显著提升,且几乎不增加计算开销。代码已公开于 https://github.com/xuhang07/TEP-Diffusion。

原文摘要 · Abstract (English)

Test-time scaling (TTS) aims to achieve better results by increasing random sampling and evaluating samples based on rules and metrics. However, in text-to-image(T2I) diffusion models, most related works focus on search strategies and reward models, yet the impact of the stochastic characteristic of noise in T2I diffusion models on the method's performance remains unexplored. In this work, we analyze the effects of randomness in T2I diffusion models and explore a new format of randomness for TTS: text embedding perturbation, which couples with existing randomness like SDE-injected noise to enhance generative diversity and quality. We start with a frequency-domain analysis of these formats of randomness and their impact on generation, and find that these two randomness exhibit complementary behavior in the frequency domain: spatial noise favors low-frequency components (early steps), while text embedding perturbation enhances high-frequency details (later steps), thereby compensating for the potential limitations of spatial noise randomness in high-frequency manipulation. Concurrently, text embedding demonstrates varying levels of tolerance to perturbation across different dimensions of the generation process. Specifically, our method consists of two key designs: (1) Introducing step-based text embedding perturbation, combining frequency-guided noise schedules with spatial noise perturbation. (2) Adapting the perturbation intensity selectively based on their frequency-specific contributions to generation and tolerance to perturbation. Our approach can be seamlessly integrated into existing TTS methods and demonstrates significant improvements on multiple benchmarks with almost no additional computation. Code is available at \href{https://github.com/xuhang07/TEP-Diffusion}{https://github.com/xuhang07/TEP-Diffusion}.

扩散模型文本生成图像生成测试时扩展

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。