研究过参数下退化优化问题的临界点，揭示其几何本质
Critical Points of Degenerate Metrics on Algebraic Varieties: A Tale of Overparametrization
- 通过投影将退化问题转化为非退化问题
- 在高度退化时，临界点分布由投影的分歧轨迹决定
- 提供计算射影流形上临界点数的工具，适用于深度学习场景
我们研究了在代数簇上定义的退化二次目标函数的临界点问题。该情形出现在机器学习中数据量相对于模型规模较小时，即通常所说的过参数化。我们的主要结果通过一个投影将退化优化问题与非退化问题联系起来。在高度退化的情况下，投影的分歧轨迹起核心作用。此外，我们提供了计数射影流形上临界点数量的工具，并讨论了深度学习中出现的具体情形。本工作将代数几何工具与机器学习思想相连接，将欧几里得距离次数的研究扩展到退化设置。
原文摘要 · Abstract (English)
We study the critical points over an algebraic variety of an optimization problem defined by a quadratic objective that is degenerate. This scenario arises in machine learning when the dataset size is small with respect to the model, and is typically referred to as overparametrization. Our main result relates the degenerate optimization problem to a nondegenerate one via a projection. In the highly-degenerate regime, we find that a central role is played by the ramification locus of the projection. Additionally, we provide tools for counting the number of critical points over projective varieties, and discuss specific cases arising from deep learning. Our work bridges tools from algebraic geometry with ideas from machine learning, and it extends the line of literature around the Euclidean distance degree to the degenerate setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。