提出相对信息增益，统一优化采样复杂度与泛化误差分析
Relative Information Gain and Gaussian Process Regression
- 引入相对信息增益衡量噪声敏感性，衔接有效维数与信息增益
- 证明其增长速率与有效维数一致，实现最优收敛率
- 在高斯过程回归中导出新风险界，适合理论学习者
在再生核希尔伯特空间中估计或最大化未知函数的样本复杂度，与有效维数及核相关的信息增益密切相关。尽管信息增益具有信息论解释优势，有效维数通常给出更优的收敛速率。本文提出一种新量——相对信息增益，用于衡量信息增益对观测噪声的敏感性。我们证明该量在有效维数与信息增益之间平滑插值，且其增长速率与有效维数相同。论文后半部分给出了高斯过程回归的新PAC-Bayesian过风险界，其中相对信息增益自然出现在复杂度项中。我们基于核的谱特性建立了相对信息增益的上界，结合该风险界可得极小极大最优收敛速率。
原文摘要 · Abstract (English)
The sample complexity of estimating or maximising an unknown function in a reproducing kernel Hilbert space is known to be linked to both the effective dimension and the information gain associated with the kernel. While the information gain has an attractive information-theoretic interpretation, the effective dimension typically results in better rates. We introduce a new quantity called the relative information gain, which measures the sensitivity of the information gain with respect to the observation noise. We show that the relative information gain smoothly interpolates between the effective dimension and the information gain, and that the relative information gain has the same growth rate as the effective dimension. In the second half of the paper, we prove a new PAC-Bayesian excess risk bound for Gaussian process regression. The relative information gain arises naturally from the complexity term in this PAC-Bayesian bound. We prove bounds on the relative information gain that depend on the spectral properties of the kernel. When these upper bounds are combined with our excess risk bound, we obtain minimax-optimal rates of convergence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。