大带宽下高斯RBF核空间趋近欧氏空间,主成分分析结果接近线性方法。
Data eccentricity, asymptotics of Gaussian RBF reproducing kernel Hilbert space, and kernel PCA

- 在大带宽极限下,高斯RBF核空间与欧氏空间渐近等距
- 核PCA的特征值、投影和主成分均收敛到线性PCA结果
- 数据几何偏心率ρ可预测不同数据集的收敛速度
我们证明,在各向同性缩放下,高斯RBF再生核希尔伯特空间(RKHS)在带宽趋于无穷时渐近等距于欧氏空间。这强烈暗示依赖于RKHS度量性质的核方法,在大带宽下将表现出与线性核相似的结果。这一视角有助于理解高斯核中心化核相关性(CKA)的渐近行为。进一步研究核PCA,发现当带宽σ→∞时,高斯RBF的特征值、特征投影及主成分均收敛至经典线性PCA结果。对于给定数据表示,两类核的特征嵌入与正交主成分基仅相差一个几何相似变换,残差大小为O((ρ/σ)²),其中ρ是数据表示的几何偏心率,定义为样本间最大距离与中位数距离之比。在多种数据集上的实验表明,ρ能作为数据集特定收敛行为的简单可靠预测指标。
原文摘要 · Abstract (English)
We show that, up to isotropic scaling, the Gaussian RBF reproducing kernel Hilbert space (RKHS) is asymptotically isometric to Euclidean space in the large bandwidth limit. This strongly suggests that kernel-based constructions reliant on metric properties of the RKHS will yield results for Gaussian RBF kernels that similarly approach those of linear kernels for large bandwidths. The asymptotic behavior of Gaussian CKA can be understood in this light. We further consider kernel PCA, showing that Gaussian RBF eigenvalues, eigenprojections, and principal components all converge to those of classical (linear) PCA as bandwidth $σ\rightarrow \infty$. For a given data representation, both the RKHS feature embeddings and the orthogonal PCA eigenframes of the two kernel types differ asymptotically by a geometric similarity transformation, up to a residual of size $O \left (\fracρσ \right )^2$, where $ρ$ is a measure of geometric eccentricity of the representation, equal to the ratio of maximum to median pairwise distance between data examples. Experiments over a diverse collection of data sets demonstrate that $ρ$ provides a simple and reliable predictor of dataset-specific convergence behavior in the top principal directions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。