用代数框架解释高秩神经网络为何泛化好
Why High-rank Neural Networks Generalize?: An Algebraic Framework with RKHSs
- 引入柯普曼算子与再生核希尔伯特空间构建新理论框架
- 推导出适用于更广泛模型的新型径向复杂度上界
- 为实际神经网络泛化能力提供新解释，适合理论研究者
本文利用柯普曼算子、群表示和再生核希尔伯特空间（RKHS），为深度神经网络推导出一种新的径向复杂度上界。该上界揭示了具有高秩权重矩阵的模型为何能良好泛化。尽管已有类似上界试图解释此现象，但仅适用于有限类型的模型。本文提出神经网络的代数表示及对应的核函数，构建适用于更广泛真实模型的RKHS，从而拓展了基于柯普曼算子的径向复杂度理论在实际场景中的适用性。
原文摘要 · Abstract (English)
We derive a new Rademacher complexity bound for deep neural networks using Koopman operators, group representations, and reproducing kernel Hilbert spaces (RKHSs). The proposed bound describes why the models with high-rank weight matrices generalize well. Although there are existing bounds that attempt to describe this phenomenon, these existing bounds can be applied to limited types of models. We introduce an algebraic representation of neural networks and a kernel function to construct an RKHS to derive a bound for a wider range of realistic models. This work paves the way for the Koopman-based theory for Rademacher complexity bounds to be valid for more practical situations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。