arXiv:2604.12827cs.LGcs.AI2026-04

改进随机特征模型的误差预测,揭示协方差项对泛化差距的关键影响。

Loop Corrections in Random Feature Models: Training Error and Generalization Gap

论文配图:Loop Corrections in Random Feature Models: Training Error and Generalization Gap
图 1 · 摘自论文原文
  • 基于有限再生核恒等式推导出协方差修正项,避免强假设
  • 协方差项消除宽度倒数偏差,提升训练误差预测精度
  • 适用于固定特征、非学习场景下的理论分析,适合研究泛化机制

我们研究了固定设计下随机特征岭回归在均值核近似之外的表现。期望取自冻结特征集合,条件于训练样本。由于预测器是经验核的非线性函数,其均值训练误差、测试误差及条件泛化差距依赖于中心化核协方差与均值核对象。通过有限再生核恒等式导出协方差级(一环)修正项,避免了几乎必然的Neumann级数假设,并给出训练误差的显式余项界。测试误差修正还需混合训练-测试协方差张量,因此一般无法仅从训练受限核律恢复。数值实验表明,在受控正则化范围内,协方差项可消除主导的反宽度偏差;而舍弃混合协方差的消融实验虽不影响训练修正,却使测试误差预测失效。实验还识别出一个由再生核波动诊断界定的宽度-正则化边界,超出该边界时二阶截断失准。所有反宽度结论均针对固定样本量、深度、设计及正核级正则化。分析基于冻结特征,未建模特征学习或神经切线核演化。

原文摘要 · Abstract (English)

We study fixed-design random feature ridge regression beyond the mean-kernel approximation. The expectation is taken over the frozen-feature ensemble, conditional on the training sample. Because the predictor is a nonlinear function of the empirical kernel, its mean training error, test error, and conditional generalization gap depend on centered kernel covariances as well as on mean kernel objects. We derive the covariance-level, or one-loop, corrections from a finite resolvent identity. This avoids an almost-sure Neumann-series assumption and yields an explicit remainder bound for the training error. The test correction additionally requires mixed train--test covariance tensors and therefore cannot, in general, be recovered from the train-restricted kernel law alone. Numerical experiments show that the covariance term removes the leading inverse-width discrepancy in a controlled regularization regime, while an ablation that discards the mixed train--test covariance fails for the test error even though the training correction is unchanged. They also identify a width--regularization boundary, measured by a resolvent-fluctuation diagnostic, beyond which the second-order truncation loses accuracy. All inverse-width statements are for fixed sample size, depth, design, and positive kernel-level regularization. The analysis concerns frozen features and does not model feature learning or neural-tangent-kernel evolution.

随机特征泛化分析误差预测协方差修正

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。