首次给出非线性ICA的有限样本分析,明确所需样本量与可识别性边界。
Finite-Sample Analysis of Nonlinear Independent Component Analysis:Sample Complexity and Identifiability Bounds
- 建立泛化误差与识别误差的直接关系,避免参数空间分析导致的性能下降
- 证明信息论下界,验证样本复杂度结果的最优性
- 在实际SGD优化下仍保持相同样本效率,适合关注训练可靠性的研究者
独立成分分析(ICA)是一种基础的无监督学习技术,通过分离混合信号来揭示数据中的潜在结构。尽管非线性ICA的渐近可识别性已有大量研究,但其学习算法的有限样本统计性质仍不清晰,给实践者确定可靠源恢复所需的样本量带来挑战。本文对使用神经网络编码器的非线性ICA进行了全面的有限样本分析,首次实现上下界匹配的完整表征。理论贡献包括:第一,建立过剩风险与识别误差之间的直接联系,绕过参数空间论证,避免率退化带来的次优缩放;第二,证明匹配的信息论下界,确认样本复杂度结果的最优性;第三,将分析扩展至实际的SGD优化,表明在标准景观假设下,有限次梯度下降亦可实现相同的样本效率。通过精心设计的模拟实验验证了理论预测。该工作揭示了未来研究神经网络训练有限样本行为的重要方向,并强调了所验证缩放律在维度与多样性方面的关键意义。
原文摘要 · Abstract (English)
Independent Component Analysis (ICA) is a fundamental unsupervised learning technique foruncovering latent structure in data by separating mixed signals into their independent sources. While substantial progress has been made in establishing asymptotic identifiability guarantees for nonlinear ICA, the finite-sample statistical properties of learning algorithms remain poorly understood. This gap poses significant challenges for practitioners who must determine appropriate sample sizes for reliable source recovery. This paper presents a comprehensive finite-sample analysis of nonlinear ICA with neural network encoders, providing the first complete characterization with matching upper and lower bounds. Our theoretical development introduces three key technical contributions. First, we establish a direct relationship between excess risk and identification error that bypasses parameter-space arguments, thereby avoiding the rate degradation that would otherwise yield suboptimal scaling. Second, we prove matching information-theoretic lower bounds that confirm the optimality of our sample complexity results. Third, we extend our analysis to practical SGD optimization, showing that the same sample efficiency can be achieved with finite-iteration gradient descent under standard landscape assumptions. We validate our theoretical predictions through carefully designed simulation experiments. This gap points toward valuable future research on finite-sample behavior of neural network training and highlights the importance of our validated scaling laws for dimension and diversity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。