从有限样本中无偏估计核积分算子谱矩，揭示深层网络表征几何。
Estimating the Spectral Moments of the Kernel Integral Operator from Finite Sample Matrices
- 基于动态规划构建无偏谱矩估计算法
- 在RBF核上验证与理论谱一致，误差小
- 适用于分析神经网络学习表征的几何结构
当输入数量和特征维度受限时，分析数据采样特征的结构极具挑战。传统方法依赖有限测量矩阵的样本协方差矩阵特征值谱，但该谱易受测量矩阵规模影响，导致偏差。本文提出一种新算法，可在有限采样下无偏估计核积分算子在无限输入与特征极限下的谱矩。该方法基于动态规划，计算高效，可准确估计算子谱的各阶矩。我们在径向基函数（RBF）核上验证了估计算法的准确性，结果与理论谱高度一致。进一步展示了该方法在理解神经网络中学习表征几何结构方面的实用性和鲁棒性。
原文摘要 · Abstract (English)
Analyzing the structure of sampled features from an input data distribution is challenging when constrained by limited measurements in both the number of inputs and features. Traditional approaches often rely on the eigenvalue spectrum of the sample covariance matrix derived from finite measurement matrices; however, these spectra are sensitive to the size of the measurement matrix, leading to biased insights. In this paper, we introduce a novel algorithm that provides unbiased estimates of the spectral moments of the kernel integral operator in the limit of infinite inputs and features from finitely sampled measurement matrices. Our method, based on dynamic programming, is efficient and capable of estimating the moments of the operator spectrum. We demonstrate the accuracy of our estimator on radial basis function (RBF) kernels, highlighting its consistency with the theoretical spectra. Furthermore, we showcase the practical utility and robustness of our method in understanding the geometry of learned representations in neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。