通过主成分方向扰动特征,无须训练即可提升单步扩散模型的图像多样性。
STRIDE: Training-Free Diversity Guidance via PCA-Directed Feature Perturbation in Single-Step Diffusion Models

- 在单次前向传播中,沿模型自身特征主成分方向注入空间一致噪声。
- 在多个数据集上显著降低批次内相似度,同时保持高文本对齐度。
- 无需训练或优化,适合实时生成场景中的多样性增强需求。
蒸馏后的单步(T=1)或少步(T≤4)扩散模型支持实时图像生成,但样本多样性通常低于多步模型。多步扩散中可通过调度、轨迹或迭代优化引入多样性,但在少步或单步设置下这些机制不可用,限制了现有多样性增强方法的效果。直接扰动中间特征往往无效,要么增益有限,要么损害生成质量。本文提出STRIDE,一种无需训练和优化的方法,在单次前向传播中将空间一致的噪声注入中间Transformer特征,并投影至模型自身激活的主成分方向,确保扰动位于学习到的特征流形上。该设计使表示空间中的变化可控且有意义。在FLUX.1-schnell和SD3.5 Turbo上,覆盖COCO、DrawBench、PartiPrompts和GenEval等多个数据集的实验表明,STRIDE持续提升多样性,同时保持强文本对齐。尤其在减少批次内相似度的同时,对CLIP分数影响极小,且在多样性-保真度权衡上优于现有训练自由基线。结果表明,在缺乏迭代精炼的情况下,提升少步与单步扩散模型的多样性不在于增强扰动强度,而在于对齐内部表示结构。
原文摘要 · Abstract (English)
Distilled one-step (T=1) or few-step (T$\leq$4) diffusion models enable real-time image generation but often exhibit reduced sample diversity compared to their multi-step counterparts. In multi-step diffusion, diversity can be introduced through schedules, trajectories, or iterative optimization; however, these mechanisms are unavailable in the few-step or single-step setting, limiting the effectiveness of existing diversity-enhancing methods. A natural alternative is to perturb intermediate features, but naive feature perturbation is often ineffective, either yielding limited diversity gains or degrading generation quality. We argue that effective diversity injection in few-step models requires perturbations that respect the model's learned feature geometry. Based on this insight, we propose STRIDE, a training-free and optimization-free method that operates in a single forward pass. STRIDE injects spatially coherent (pink) noise into intermediate transformer features, projected onto the principal components of the model's own activations, ensuring that perturbations lie on the learned feature manifold. This design enables controlled variation along meaningful directions in the representation space. Extensive experiments on FLUX.1-schnell and SD3.5 Turbo across COCO, DrawBench, PartiPrompts, and GenEval show that STRIDE consistently improves diversity while maintaining strong text alignment. In particular, STRIDE reduces intra-batch similarity with minimal impact on CLIP score, and Pareto-dominates existing training-free baselines on the diversity-fidelity frontier. These results highlight that, in the absence of iterative refinement, improving diversity in few-step and one-step diffusion depends not on increasing perturbation strength, but on aligning perturbations with the model's internal representation structure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。