揭示非均匀数据下相位恢复的三阶段学习动态与缩放规律
Fast Escape, Slow Convergence: Learning Dynamics of Phase Retrieval under Power-Law Data
- 构建降维模型，解析非各向同性数据下的多尺度演化机制
- 发现快速逃逸、缓慢收敛、低方差方向学习三阶段，误差下降遵循幂律
- 适用于研究复杂数据分布下深度学习的训练效率与收敛行为
缩放定律描述了学习性能随数据量、算力或训练时间的增长规律，已成为现代深度学习的核心主题。本文研究一个典型非线性模型：输入为各向异性高斯分布的相位恢复问题，其协方差谱服从幂律。不同于各向同性情形下动力学退化为二维系统，非各向同性导致无限层级耦合方程控制统计量演化。我们提出可处理的降维方法，揭示三阶段轨迹：(i) 低对齐状态下的快速逃逸；(ii) 统计量的缓慢收敛；(iii) 低方差方向上的谱尾学习。由此推导出均方误差的显式缩放律，表明谱衰减决定收敛时间和误差曲线。实验验证了预测的阶段和指数。这是首次在非各向同性数据的非线性回归中严格刻画缩放规律，凸显了非各向同性对学习动力学的根本重塑。
原文摘要 · Abstract (English)
Scaling laws describe how learning performance improves with data, compute, or training time, and have become a central theme in modern deep learning. We study this phenomenon in a canonical nonlinear model: phase retrieval with anisotropic Gaussian inputs whose covariance spectrum follows a power law. Unlike the isotropic case, where dynamics collapse to a two-dimensional system, anisotropy yields a qualitatively new regime in which an infinite hierarchy of coupled equations governs the evolution of the summary statistics. We develop a tractable reduction that reveals a three-phase trajectory: (i) fast escape from low alignment, (ii) slow convergence of the summary statistics, and (iii) spectral-tail learning in low-variance directions. From this decomposition, we derive explicit scaling laws for the mean-squared error, showing how spectral decay dictates convergence times and error curves. Experiments confirm the predicted phases and exponents. These results provide the first rigorous characterization of scaling laws in nonlinear regression with anisotropic data, highlighting how anisotropy reshapes learning dynamics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。