提出SPARKE方法，让扩散模型在不同提示下生成更多样且新颖的图像。
SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Score
- 基于条件熵动态调节多样性，实现提示感知的生成控制。
- 计算复杂度从O(n³)降至O(n)，支持数千次生成迭代。
- 适合需要高多样性的文本到图像生成任务，如创意设计。
扩散模型在高质量图像生成和提示引导生成方面表现卓越，但如何在提示跨度广泛时确保生成样本的充分多样性仍是一大挑战，尤其需在语义相近提示间进行提示感知的多样性评估。现有方法通过引入多样性度量进行引导以增强生成多样性。本文提出可扩展的提示感知瑞尼核熵多样性引导方法（SPARKE），利用条件熵实现对相似提示的动态多样性测量，从而支持提示感知的多样性调控。尽管基于熵的引导提升了提示感知多样性，但其依赖矩阵型熵评分导致大规模生成场景下计算成本高昂。为此，我们聚焦于条件潜在瑞尼核得分引导的特例，将熵计算与梯度优化复杂度从一般熵度量的O(n³)降低至O(n)，使多样引导采样可在数千次不同提示生成中高效执行。我们在多个文本到图像扩散模型上数值验证了SPARKE方法，结果表明该方法在不增加显著计算开销的前提下，有效提升了生成数据的提示感知多样性。代码已公开于项目主页：https://mjalali.github.io/SPARKE
原文摘要 · Abstract (English)
Diffusion models have demonstrated remarkable success in high-fidelity image synthesis and prompt-guided generative modeling. However, ensuring adequate diversity in generated samples of prompt-guided diffusion models remains a challenge, particularly when the prompts span a broad semantic spectrum and the diversity of generated data needs to be evaluated in a prompt-aware fashion across semantically similar prompts. Recent methods have introduced guidance via diversity measures to encourage more varied generations. In this work, we extend the diversity measure-based approaches by proposing the Scalable Prompt-Aware Rény Kernel Entropy Diversity Guidance (SPARKE) method for prompt-aware diversity guidance. SPARKE utilizes conditional entropy for diversity guidance, which dynamically conditions diversity measurement on similar prompts and enables prompt-aware diversity control. While the entropy-based guidance approach enhances prompt-aware diversity, its reliance on the matrix-based entropy scores poses computational challenges in large-scale generation settings. To address this, we focus on the special case of Conditional latent RKE Score Guidance, reducing entropy computation and gradient-based optimization complexity from the $O(n^3)$ of general entropy measures to $O(n)$. The reduced computational complexity allows for diversity-guided sampling over potentially thousands of generation rounds on different prompts. We numerically test the SPARKE method on several text-to-image diffusion models, demonstrating that the proposed method improves the prompt-aware diversity of the generated data without incurring significant computational costs. We release our code on the project page: https://mjalali.github.io/SPARKE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。