arXiv:2605.17212cs.LG2026-05

提出可随时验证的密度比网络,在分布偏移下实现可靠泛化保证。

Anytime PAC-Bayes for Constrained Density-Ratio Networks under Covariate Shift

论文配图:Anytime PAC-Bayes for Constrained Density-Ratio Networks under Covariate Shift
图 1 · 摘自论文原文
  • 用约束密度比网络逼近样本权重,结合即时PAC-Bayes生成泛化界。
  • 在真实数据上实现0/1损失降低,且覆盖率达99%以上,仅一次失败。
  • 适合需要严格泛化保障的高风险场景,如医疗或金融预测。

本文提出统一框架,用于处理协变量偏移下的学习问题。通过约束密度比网络近似Radon-Nikodym导数 $r^ullet = dP/dQ$,并生成即时PAC-Bayes泛化证书。基于变测度恒等式,目标风险与加权源风险之差被分解为由 $ rm{r_θ - r^ullet}_{L^2(Q)}$ 控制的比率偏差项和由加权损失方差决定的泛化差距项。通过增广拉格朗日法强制实施归一化与矩匹配的硬积分约束,并以二阶矩惩罚控制有效样本量。在固定时间下实例化PAC-Bayes于加权风险,得到伯努利-KL界,识别出网络加权吉布斯后验为唯一KL正则化最小化器,并量化了对 $L^2(Q)$ 摄动的稳定性;再通过几何剥皮强化为任意时间证书,对所有 $t \geq t_{\min}$ 一致成立。预注册的双阶段协议(包括针对解析真值的片段测试与真实数据部署)验证该框架:网络生成校准的比例,降低目标0/1损失,优于未加权ERM与经典直接比例估计基线,并达到即时证书。仅记录一次固定时间覆盖失败,各分拆覆盖率与标签偏移大小一一对应,证实协变量假设在操作上是紧的,而非证书缺陷。

原文摘要 · Abstract (English)

A unified framework for learning under covariate shift is presented, in which a constrained density-ratio network approximates the Radon-Nikodym derivative $r^\star = dP/dQ$ and feeds an anytime PAC-Bayes generalization certificate. A change-of-measure identity decomposes the gap between target risk and importance-weighted source risk into a ratio-bias term governed by $\|r_θ- r^\star\|_{L^2(Q)}$ and a generalization-gap term governed by the variability of the weighted loss. Normalization and moment-matching identities are enforced as hard integral constraints through an augmented-Lagrangian scheme, with a second-moment penalty controlling the effective sample size. PAC-Bayes is instantiated on the weighted risk in a fixed-time regime that yields Bernoulli-KL bounds, identifies the network-weighted Gibbs posterior as the unique KL-regularized minimizer, and quantifies stability under $L^2(Q)$ perturbations of the learned ratio, and is then strengthened by geometric peeling to an anytime certificate uniform in $t \geq t_{\min}$. A pre-registered two-campaign protocol combining a patch test against analytic ground truth with a real-data deployment validates the framework: the network produces calibrated ratios, reduces target $0/1$ loss against unweighted ERM and classical direct ratio-estimation baselines, and attains the anytime certificate. A single fixed-time coverage failure is recorded, with per-split coverage aligning one-to-one with the magnitude of the label shift, confirming that the covariate-only assumption is operationally tight rather than a defect of the certificate.

分布偏移泛化保证密度比估计即时证书

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。