arXiv:2602.19126cs.LGmath.PR2026-02

提出鲁棒贝叶斯随机特征模型,提升预测不确定性可靠性。

Robust Predictive Uncertainty and Double Descent in Contaminated Bayesian Random Features

  • 用霍伯污染集建模先验与似然的不匹配,采用悲观广义更新。
  • 推导出前后验预测密度的上下界,保持双下降相位结构。
  • 适合关注模型不确定性、鲁棒性与贝叶斯推断的科研人员。

我们提出一种鲁棒的贝叶斯随机特征(RF)回归方法,通过霍伯风格的污染集显式建模先验和似然的误设。基于岭正则化随机特征训练与高斯先验/似然下贝叶斯推断的经典等价性,将单一先验和似然替换为ε-和η-污染的可信集,并使用悲观广义贝叶斯更新进行推断。推导出可计算的前后验预测密度上下界,表明在适度污染下,先验与似然的模糊性会直接导致后验预测分布的污染,形成经典高斯预测的不确定性包络。引入不精确最高密度区域(IHDR)实现鲁棒预测不确定性量化,并证明其可通过调整后的高斯可信区间高效近似。进一步获得预测方差的上下界(在温和截断近似下),并证明其保留了随机特征模型已知的渐近比例增长特性。这些结果建立了贝叶斯随机特征的鲁棒性理论:预测不确定性保持计算可处理性,继承经典双下降相位结构,并在有界先验与似然误设下获得最坏情况保证。

原文摘要 · Abstract (English)

We propose a robust Bayesian formulation of random feature (RF) regression that accounts explicitly for prior and likelihood misspecification via Huber-style contamination sets. Starting from the classical equivalence between ridge-regularized RF training and Bayesian inference with Gaussian priors and likelihoods, we replace the single prior and likelihood with $ε$- and $η$-contaminated credal sets, respectively, and perform inference using pessimistic generalized Bayesian updating. We derive explicit and tractable bounds for the resulting lower and upper posterior predictive densities. These bounds show that, when contamination is moderate, prior and likelihood ambiguity effectively acts as a direct contamination of the posterior predictive distribution, yielding uncertainty envelopes around the classical Gaussian predictive. We introduce an Imprecise Highest Density Region (IHDR) for robust predictive uncertainty quantification and show that it admits an efficient approximation via an adjusted Gaussian credible interval. We further obtain predictive variance bounds (under a mild truncation approximation for the upper bound) and prove that they preserve the leading-order proportional-growth asymptotics known for RF models. Together, these results establish a robustness theory for Bayesian random features: predictive uncertainty remains computationally tractable, inherits the classical double-descent phase structure, and is improved by explicit worst-case guarantees under bounded prior and likelihood misspecification.

贝叶斯推断不确定性量化随机特征

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。