研究幂律数据下核岭回归的谱特性与泛化能力，发现有效维度决定样本需求。
Kernel ridge regression under power-law data: spectrum and generalization
- 推导多项式内积核在幂律协方差下的精确谱结构
- 证明高维下过拟合风险由数据有效维度决定而非总维数
- 首次严格分析幂律非线性核回归，适合机器学习理论研究者
本文研究独立同分布高斯数据上高维核岭回归（KRR）的性质，数据具有各向异性幂律协方差。该设定与传统核岭回归的源与容量条件有本质区别，通常假设的是核特征值谱满足幂律。我们的贡献有两点：首先，针对多项式内积核，我们给出了核谱的显式刻画，精确描述了核特征值如何继承数据的衰减规律；其次，对具有此谱行为的特定核，在高维极限下进行了过剩风险的渐近分析，表明样本复杂度由数据的有效维度决定，而非环境维度。这些结果揭示了使用幂律各向异性数据进行学习相比各向同性数据的根本优势。据我们所知，这是首个关于幂律数据下非线性核岭回归的严格分析。
原文摘要 · Abstract (English)
In this work, we investigate high-dimensional kernel ridge regression (KRR) on i.i.d. Gaussian data with anisotropic power-law covariance. This setting differs fundamentally from the classical source & capacity conditions for KRR, where power-law assumptions are typically imposed on the kernel eigen-spectrum itself. Our contributions are twofold. First, we derive an explicit characterization of the kernel spectrum for polynomial inner-product kernels, giving a precise description of how the kernel eigen-spectrum inherits the data decay. Second, we provide an asymptotic analysis of the excess risk in the high-dimensional regime for a particular kernel with this spectral behavior, showing that the sample complexity is governed by the effective dimension of the data rather than the ambient dimension. These results establish a fundamental advantage of learning with power-law anisotropic data over isotropic data. To our knowledge, this is the first rigorous treatment of non-linear KRR under power-law data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。