分析球面上浅层ReLU网络的条件数与特征谱,揭示其逼近能力与数值稳定性关系。
Condition Numbers and Eigenvalue Spectra of Shallow Networks on Spheres
- 基于抗对称准均匀采样,推导出质量与刚度矩阵的精确条件数估计。
- 给出特征值全谱的渐近估计,最小和最大特征值分别对应低阶与高阶多项式基。
- 为网络逼近性能与数值稳定性的关联提供理论依据,适合研究深度学习几何性质者阅读。
我们对定义在单位球面 $\bS^d$ 上的浅层 ReLU$^k$ 神经网络所生成的质量矩阵与刚度矩阵的条件数进行了估计。当采样点集 $\\{θ_j^*\ }_{j=1}^n \subset \bS^d$ 为抗对称准均匀时,条件数达到精确上界。在此情况下,我们得到了特征值全谱的精确渐近估计,并刻画了相应特征空间的结构:最小特征值对应低阶多项式基,最大特征值则与高阶多项式相关。该谱分析建立了网络逼近能力与数值稳定性之间的精确对应关系。
原文摘要 · Abstract (English)
We present an estimation of the condition numbers of the \emph{mass} and \emph{stiffness} matrices arising from shallow ReLU$^k$ neural networks defined on the unit sphere~$\mathbb{S}^d$. In particular, when $\{θ_j^*\}_{j=1}^n \subset \mathbb{S}^d$ is \emph{antipodally quasi-uniform}, the condition number is sharp. Indeed, in this case, we obtain sharp asymptotic estimates for the full spectrum of eigenvalues and characterize the structure of the corresponding eigenspaces, showing that the smallest eigenvalues are associated with an eigenbasis of low-degree polynomials while the largest eigenvalues are linked to high-degree polynomials. This spectral analysis establishes a precise correspondence between the approximation power of the network and its numerical stability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。