研究深度矩阵分解中低秩隐式正则的稳定性，揭示噪声下优化轨迹的鲁棒性。
Stability of Low-Rank Implicit Regularization in Perturbed Deep Matrix Factorization
- 通过谱条件分析无噪情形下的低秩区间，明确目标谱、初始化与步长的协同作用。
- 证明扰动下梯度下降收敛性，量化扰动大小对迭代次数和特征值恢复的影响。
- 发现扰动后低秩相位仍稳定，有效秩接近无噪时的L阶近似，依赖扰动规模。
本文研究深度矩阵分解中低秩隐式正则的稳定性，该模型可帮助理解基于梯度的训练如何偏好低复杂度结构。首先回顾无噪情形，推导出梯度下降呈现非空低秩区间的充分谱条件，阐明目标谱、初始化与步长共同决定低秩相位是否在优化轨迹中可观测。随后分析含加性扰动的目标矩阵情形，通过在特征值层面研究扰动梯度下降动力学，证明收敛性并量化扰动大小对迭代复杂度和特征值恢复的影响。最终建立扰动下低秩相位的稳定性：在扰动低秩区间内，迭代解的有效秩保持接近无噪目标的L阶近似，其依赖关系明确给出。数值实验验证了理论预测，并展示了谱结构在决定该稳定性出现时机中的作用。
原文摘要 · Abstract (English)
This paper studies the stability of low-rank implicit regularization in deep matrix factorization, a tractable model for understanding how gradient-based training can favor low-complexity structure. We first revisit the noiseless setting and derive sufficient spectral conditions under which gradient descent exhibits a nonempty low-rank interval. These conditions clarify how the target spectrum, initialization, and step size jointly determine when a low-rank phase is observable along the optimization trajectory. We then analyze the perturbed problem, where the target matrix is subject to an additive perturbation. By studying the perturbed gradient descent dynamics at the eigenvalue level, we prove convergence guarantees and quantify how the perturbation size affects iteration complexity and eigenvalue recovery. Finally, we establish stability of the low-rank phase under perturbation: the effective rank of the iterates remains close to that of the rank-L approximation of the noiseless target over a perturbed low-rank interval, with explicit dependence on the perturbation size. Numerical illustrations support the theoretical predictions and illustrate the role of spectral structure in determining when this stability is observed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。