通过种子调控提升扩散模型生成一致性与质量。
Determinism of Randomness: Prompt-Residual Seed Shaping for Diffusion Generation
- 用提示词残差作为代理,仅注入切向分量来优化初始噪声。
- 在多个基准上显著提升生成内容与提示词的对齐度和视觉质量。
- 适合关注生成稳定性和可控性的研究人员或应用开发者。
扩散模型从各向同性的高斯潜空间开始生成,但仅改变随机种子便可能导致提示词契合度、构图和视觉质量的显著差异。本文通过分析初始噪声到生成语义的语义映射,揭示了采样流虽局部可逆,但后续语义投影为多对一,导致潜空间中存在退化的拉回半度量:大部分局部方向几乎语义不变,而敏感变化集中在更小的水平子空间。这为种子彩票现象提供了几何解释。受此启发,我们提出一种无需训练的提示词残差种子调控方法。该方法不试图恢复精确水平空间,而是使用单个高噪声冷启动提示残差作为模型耦合代理,仅注入其切向分量,并将种子收缩回原始高斯半径球面。该方法保持初始化先验兼容性,且只需在标准采样前添加一次条件/非条件探测。在多个生成基准测试中,该方法相比标准采样在对齐度和质量指标上均有提升,验证了代理的有效性及语义各向异性的解释价值。
原文摘要 · Abstract (English)
Diffusion models start generation from an isotropic Gaussian latent, yet changing only the random seed can lead to large differences in prompt faithfulness, composition, and visual quality. We study this seed sensitivity through the semantic map from initial noise to generated meaning. Although the sampling flow is locally invertible, the subsequent semantic projection is many-to-one, inducing a degenerate pullback semi-metric on the latent space: most local directions are nearly semantic-invariant, while semantic-sensitive variation is concentrated in a much smaller horizontal subspace. This provides an explanatory geometric view of the seed lottery. Motivated by this view, we introduce a training-free prompt-residual seed-shaping procedure. Rather than claiming to recover the exact horizontal space, the method uses a single high-noise cold-start prompt residual as a model-coupled proxy, injects only its tangential component, and retracts the seed to the original Gaussian radius shell. This keeps the initialization prior-compatible while adding only one conditional/unconditional probe before standard sampling. Across multiple generation benchmarks, the method improves alignment and quality metrics over standard sampling, supporting both the practical value of the proxy and the explanatory relevance of semantic anisotropy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。