提出RICA方法,用局部几何结构解释无生成假设下的表征解耦。
Disentanglement Beyond Generative Models with Riemannian ICA

- 用黎曼几何替代全局生成模型,从数据点出发分析局部解耦结构。
- 引入解耦张量,通过数据对数似然的海森矩阵与里奇曲率刻画点级解耦。
- 在多流形上成功恢复真实源信号,优于依赖坐标系的ICA基线方法。
现有解耦理论多基于生成模型假设,要求潜在变量统计独立,但现代预训练编码器在无生成假设下仍能学习到解耦特征,缺乏统一理论解释。本文提出黎曼独立成分分析(RICA),将传统ICA的全局生成模型替换为局部几何结构。核心思想是:每个数据点的变因可通过从该点出发的径向曲线映射到潜在空间中的轴对齐线来理解。利用黎曼几何形式化这一观点,提出解耦张量,量化点级解耦程度,其依赖于数据对数似然的海森矩阵及模型诱导的里奇曲率。在已知真实源的受控源恢复任务中,RICA可在多个流形上成功恢复源信号,而ICA基线性能高度依赖观测表示的坐标系。本工作为无需全局生成假设的局部解耦提供了理论基础。
原文摘要 · Abstract (English)
There is a gap between the theoretical foundations of disentanglement and the practice of modern representation learning. Existing theoretical frameworks, particularly Independent Component Analysis (ICA) and its nonlinear variants, assume a generative model with statistically independent latent variables underlying the data so that disentanglement amounts to identifying the latents that could have generated the data. This generative framework is interpretable and theoretically justified, but its strong assumptions make it difficult to apply to modern representation learning. Modern pretrained encoders often learn features that exhibit disentangled properties without making generative assumptions, yet there is no general theory for interpreting these features as independent factors of variation. We take a step toward such a theory by introducing Riemannian ICA (RICA), which replaces ICA's global generative model with local geometric structure. RICA is founded on the observation that in ICA, the factors of variation underlying a data point can be understood through radial curves emanating from the point that map to axis-aligned lines in the latent space. We formalize this perspective using Riemannian geometry and introduce our theory in a way that is consistent with the existing generative approach. Our main contribution is the disentanglement tensor, which encodes a second-order notion of disentanglement that we call pointwise disentanglement. This tensor depends on the Hessian of the data log likelihood as well as the Ricci curvature induced by the model. In a controlled source recovery setting with known ground-truth sources, RICA recovers sources across several manifolds, while the success of ICA baselines depends on the coordinates used to represent the observations. Our work provides a theoretical basis for studying local disentanglement without assuming a global generative model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。