解析正则化深度矩阵分解的优化景观，揭示为何梯度法总能收敛到局部最优解。
A Complete Loss Landscape Analysis of Regularized Deep Matrix Factorization
- 给出所有临界点的闭式表征，明确其分类标准。
- 证明所有临界点要么是局部极小值，要么是严格鞍点。
- 理论解释梯度法为何几乎总收敛到局部最优，适合优化与深度学习研究者。
尽管深度矩阵分解（DMF）在多个领域有广泛应用，其优化基础仍不清晰。本文系统分析了正则化DMF问题的损失景观：首先给出所有临界点的闭式表征；在此基础上，精确刻画临界点为局部极小、全局极小、严格鞍点或非严格鞍点的条件；进一步推导出所有临界点均为局部极小或严格鞍点的充要条件。该结果揭示了为何基于梯度的方法几乎总能收敛至局部极小解。最后，通过数值实验可视化损失景观以验证理论。
原文摘要 · Abstract (English)
Despite its wide range of applications across various domains, the optimization foundations of deep matrix factorization (DMF) remain largely open. In this work, we aim to fill this gap by conducting a comprehensive study of the loss landscape of the regularized DMF problem. Toward this goal, we first provide a closed-form characterization of all critical points of the problem. Building on this, we establish precise conditions under which a critical point is a local minimizer, a global minimizer, a strict saddle point, or a non-strict saddle point. Leveraging these results, we derive a necessary and sufficient condition under which every critical point is either a local minimizer or a strict saddle point. This provides insights into why gradient-based methods almost always converge to a local minimizer of the regularized DMF problem. Finally, we conduct numerical experiments to visualize its loss landscape to support our theory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。