提出新方法揭示神经网络中隐藏的关键参数集，发现大样本下存在复杂临界点。
Uncovering Critical Sets of Deep Neural Networks via Sample-Independent Critical Lifting
- 设计无样本依赖的提升算子，建立网络参数间的映射关系
- 证明在足够大样本下存在依赖样本的临界点，且其中包含鞍点
- 为理解深层网络优化机制提供新视角，适合研究模型训练动态者
本文研究神经网络临界点的样本依赖性。提出一种无样本依赖的临界提升算子，将一个网络的参数与另一网络的一组参数关联起来，从而定义出样本依赖和样本无关的提升临界点。通过实例表明，先前研究的临界嵌入未能捕捉所有样本无关的提升临界点。最后，我们证明在充分大的样本量下存在样本依赖的提升临界点，并证实其中包含鞍点。
原文摘要 · Abstract (English)
This paper investigates the sample dependence of critical points for neural networks. We introduce a sample-independent critical lifting operator that associates a parameter of one network with a set of parameters of another, thus defining sample-dependent and sample-independent lifted critical points. We then show by example that previously studied critical embeddings do not capture all sample-independent lifted critical points. Finally, we demonstrate the existence of sample-dependent lifted critical points for sufficiently large sample sizes and prove that saddles appear among them.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。