通过费舍尔几何解析扩散模型在隐空间退化的原因
Understanding Latent Diffusability via Fisher Geometry
- 用MMSE变化率衡量隐空间扩散能力，分解为费舍尔信息与费舍尔信息速率
- 发现编码器与数据几何的交互决定扩散稳定性，揭示四大退化机制
- 理论给出保持扩散稳定性的条件，适用于各类自编码架构
扩散模型在隐空间中常出现性能下降，但其根本原因尚不明确。本文通过扩散轨迹上最小均方误差（MMSE）的变化率量化隐空间扩散能力，将该变化率分解为费舍尔信息（FI）和费舍尔信息速率（FIR）的贡献。研究发现，全局等距仅保证FI对齐，而FIR由编码器与数据几何的相互作用决定。分析揭示了四种导致扩散退化的惩罚项：维度压缩、切向失真、高频编码器曲率及内在数据曲率。本文推导出保持FIR稳定的理论条件，并在多种自编码架构上验证了理论边界的影响。研究建立了一个以FI和FIR为核心的完整分析框架，用于理解隐空间扩散性。
原文摘要 · Abstract (English)
Diffusion models often degrade in latent spaces, yet the formal causes remain poorly understood. We quantify latent-space diffusability via the rate of change of the Minimum Mean Squared Error (MMSE) along the diffusion trajectory. Our framework decomposes this MMSE rate into contributions from Fisher Information (FI) and Fisher Information Rate (FIR). We demonstrate that while global isometry ensures FI alignment, FIR is governed by the interplay between encoder and data geometries. Our analysis decouples diffusion degradation into four penalties: dimensional compression, tangential distortion, high-frequency encoder curvature, and intrinsic data curvature. We derive theoretical conditions for FIR preservation to ensure stable diffusability. Experiments across diverse autoencoding architectures demonstrate the implications of our theoretical bounds. We establish FI and FIR as a comprehensive analytical framework for understanding latent diffusability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。