揭示高维多组分ICA中学习与竞争的相变规律,解释为何初始值影响恢复效果。
Learnability and Competition in High-Dimensional Multi-Component ICA

- 基于平均场理论建模多组分在线ICA,捕捉学习与正交化耦合机制。
- 发现初始值导致的两种相:解耦(独立收敛)与竞争(冲突延迟)。
- 预测学习率阶梯现象:可恢复组分数随学习率突变,适用于遥感等高维数据。
独立成分分析(ICA)是无监督表示学习的基础工具,但其高维理论仍主要局限于单组分恢复。本文建立了多组分在线ICA的渐近精确平均场理论,刻画了同时学习与正交化带来的耦合效应。在高维极限下,学习估计与真实成分的联合经验分布收敛到确定性过程,导出学习方向与真实成分重叠矩阵的闭式常微分方程系统。该表征揭示了真正的多组分、初始化驱动的相结构:解耦相中估计值分别对齐不同成分并近乎独立演化;竞争相中重叠初始值引发正交驱动冲突,导致缓慢重定向与延迟收敛。稳态分析给出了显式的可学习边界与竞争条件,关联步长、数据矩和初始化。这些条件表明,更大的高阶矩与竞争会缩小稳定学习率窗口,延长收敛时间,并预测组分数随学习率呈阶梯式变化。合成数据与高光谱遥感数据实验验证了预测轨迹与相行为。
原文摘要 · Abstract (English)
Independent Component Analysis (ICA) is a foundational tool for unsupervised representation learning, yet its high-dimensional theory remains largely limited to single-component recovery. We develop an asymptotically exact mean-field theory for multi-component online ICA, capturing the coupling induced by simultaneous learning and orthogonalization. In the high-dimensional limit, the joint empirical distribution of learned estimates and ground-truth components converges to a deterministic process, yielding a closed ODE system for the overlap matrix between learned directions and true components. This characterization reveals a genuinely multi-component, initialization-driven phase structure: a decoupled regime, where estimates align with distinct components and evolve nearly independently, and a competition regime, where overlapping initializations induce orthogonality-driven conflicts, slow reorientation, and delayed convergence. Our steady-state analysis gives explicit learnability boundaries and competition conditions linking step size, data moments, and initialization. These conditions show that larger higher-order moments and competition shrink the stable learning-rate window, increase convergence times, and predict a staircase phenomenon in which the number of recoverable components changes discretely with the learning rate. Experiments on synthetic data and hyperspectral remote sensing data validate the predicted trajectories and phase behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。