研究随机函数型变量下线性回归的泛化能力与良性过拟合现象。
Generalisation and benign over-fitting for linear regression onto random functional covariates
- 在非独立数据设定下分析岭回归与无岭回归的预测性能。
- 当特征数p增速快于样本数n时，获得预测风险收敛速率。
- 揭示协变量噪声对良性过拟合的关键作用，适合理论学习者。
我们研究了在协变量向量由p个随机、均方连续函数在潜在度量空间中n个未观测位置评估所得，并受加性噪声影响的情况下，岭回归与无岭最小二乘回归的理论预测性能。该设定脱离了标准i.i.d.假设，使得n个协变量向量仅交换对称而非独立。在维度间独立性、四阶矩及其它正则性条件下，利用Barzilai和Shamir的最新结果，我们获得了适用于该随机函数型协变量设定的预测超额风险的概率界。在p相对于n合适快速增长的范围内，推导出收敛速率，揭示模型各要素如何共同决定收敛行为，以及协变量噪声在良性过拟合中的作用。
原文摘要 · Abstract (English)
We study theoretical predictive performance of ridge and ridge-less least-squares regression when covariate vectors arise from evaluating $p$ random, means-square continuous functions over a latent metric space at $n$ random and unobserved locations, subject to additive noise. This leads us away from the standard assumption of i.i.d. data to a setting in which the $n$ covariate vectors are exchangeable but not independent in general. Under an assumption of independence across dimensions, $4$-th order moment, and other regularity conditions, we obtain probabilistic bounds on a notion of predictive excess risk adapted to our random functional covariate setting, making use of recent results of Barzilai and Shamir. We derive convergence rates in regimes where $p$ grows suitably fast relative to $n$, illustrating interplay between ingredients of the model in determining convergence behaviour and the role of additive covariate noise in benign-overfitting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。