用自然语言提示自动解析扩散模型隐空间，发现隐藏语义模式。
Decoding Diffusion: A Scalable Framework for Unsupervised Analysis of Latent Space Biases and Representations Using Natural Language Prompts
- 通过自然语言提示直接映射隐空间方向，无需训练特定向量。
- 可识别多领域隐藏关联，揭示模型学习的细微表示与偏见。
- 适合研究模型可解释性、隐空间分析的研究者使用。
图像生成领域的进展使扩散模型成为生成高质量图像的强大工具。然而，其迭代去噪过程使得理解与解释其语义隐空间比其他生成模型（如GAN）更具挑战性。现有方法试图在隐空间中识别语义明确的方向，但通常依赖人工解读或受限于可训练向量数量，限制了应用范围与实用性。本文提出一种新颖的无监督框架，用于探索扩散模型的隐空间。直接利用自然语言提示与图像字幕映射隐空间方向，实现对隐藏特征的自动理解，并支持更广泛的分析，无需训练特定向量。该方法提供了更可扩展、可解释的语义知识理解方式，促进对扩散模型隐空间中偏见与细微表示的全面分析。实验表明，该框架能在多个领域中发现隐藏模式与关联，为扩散模型隐空间的可解释性提供新洞见。
原文摘要 · Abstract (English)
Recent advances in image generation have made diffusion models powerful tools for creating high-quality images. However, their iterative denoising process makes understanding and interpreting their semantic latent spaces more challenging than other generative models, such as GANs. Recent methods have attempted to address this issue by identifying semantically meaningful directions within the latent space. However, they often need manual interpretation or are limited in the number of vectors that can be trained, restricting their scope and utility. This paper proposes a novel framework for unsupervised exploration of diffusion latent spaces. We directly leverage natural language prompts and image captions to map latent directions. This method allows for the automatic understanding of hidden features and supports a broader range of analysis without the need to train specific vectors. Our method provides a more scalable and interpretable understanding of the semantic knowledge encoded within diffusion models, facilitating comprehensive analysis of latent biases and the nuanced representations these models learn. Experimental results show that our framework can uncover hidden patterns and associations in various domains, offering new insights into the interpretability of diffusion model latent spaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。