arXiv:2601.15441cs.LGcs.CV2026-01被引 3

让扩散模型的隐层向量对齐人类概念,实现可解释的图像控制。

CASL: Concept-Aligned Sparse Latents for Interpreting Diffusion Models

  • 用监督方法将隐层激活与语义概念对齐,突破无监督局限。
  • 提出的编辑精度比(EPR)指标验证了概念特异性与属性保留性提升。
  • 适合关注模型可解释性、可控生成的研究者与开发者。

扩散模型内部激活蕴含丰富语义信息,但其表征难以解释。尽管稀疏自编码器(SAEs)在解耦潜在表示方面展现潜力,现有基于SAE的方法依赖无监督方式,无法将稀疏特征与人类可理解的概念对齐,限制了对生成图像的可靠语义控制。本文提出CASL(Concept-Aligned Sparse Latents),一种监督框架,将扩散模型的稀疏潜空间与语义概念对齐。CASL首先在冻结的U-Net激活上训练SAE以获得解耦的潜表示,再学习一个轻量级线性映射,将每个概念关联到少量相关潜维度。为验证对齐方向的语义意义,我们提出CASL-Steer,一种仅用于因果探查的受控潜空间干预,沿学习到的概念轴移动激活。此外,引入编辑精度比(EPR)度量,联合评估概念特异性和无关属性的保留程度。实验表明,该方法在编辑精度与可解释性上均优于现有方法。据我们所知,这是首个在扩散模型中实现潜表示与语义概念监督对齐的工作。

原文摘要 · Abstract (English)

Internal activations of diffusion models encode rich semantic information, but interpreting such representations remains challenging. While Sparse Autoencoders (SAEs) have shown promise in disentangling latent representations, existing SAE-based methods for diffusion model understanding rely on unsupervised approaches that fail to align sparse features with human-understandable concepts. This limits their ability to provide reliable semantic control over generated images. We introduce CASL (Concept-Aligned Sparse Latents), a supervised framework that aligns sparse latent dimensions of diffusion models with semantic concepts. CASL first trains an SAE on frozen U-Net activations to obtain disentangled latent representations, and then learns a lightweight linear mapping that associates each concept with a small set of relevant latent dimensions. To validate the semantic meaning of these aligned directions, we propose CASL-Steer, a controlled latent intervention that shifts activations along the learned concept axis. Unlike editing methods, CASL-Steer is used solely as a causal probe to reveal how concept-aligned latents influence generated content. We further introduce the Editing Precision Ratio (EPR), a metric that jointly measures concept specificity and the preservation of unrelated attributes. Experiments show that our method achieves superior editing precision and interpretability compared to existing approaches. To the best of our knowledge, this is the first work to achieve supervised alignment between latent representations and semantic concepts in diffusion models.

扩散模型可解释性潜空间语义控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。