提出可解释的风险变化分析框架,诊断模型在数据偏移下的性能波动。
TRACE: Theoretical Risk Attribution under Covariate-shift Effects
- 将风险变化分解为四个可操作因子:泛化差距、模型变更惩罚、数据偏移惩罚
- 通过梯度敏感性和最优传输/最大均值差异量化数据偏移,精度达95%以上
- 生成部署门控评分,支持高效安全的模型替换决策
当源域训练模型 $Q$ 被在偏移数据上训练的新模型 $ ilde{Q}$ 替代时,其在源域上的性能可能不可预测地改变。本文研究协变量偏移下两模型风险变化 $ΔR := R_P( ilde{Q}) - R_P(Q)$,提出 TRACE(Theoretical Risk Attribution under Covariate-shift Effects)框架,将 $|ΔR|$ 分解为可解释的上界。该分解将风险变化拆分为两个泛化差距、一个模型变更惩罚和一个协变量偏移惩罚,使边界成为强大的诊断工具。我们实例化各项:协变量偏移惩罚通过高分位数输入梯度敏感性与数据偏移度量估计,采用默认特征空间最优传输(OT),并提供基于最大均值差异(MMD)的鲁棒替代方案;模型变更惩罚由目标样本上两模型输出平均距离控制;泛化差距在保留数据上估计。在理想线性回归设置中,TRACE 边界正确捕捉了真实风险差随偏移幅度的缩放关系。在合成及视觉基准测试中,TRACE 诊断有效且与真实性能退化保持强单调关系。关键的是,我们推导出部署门控评分,与 $|ΔR|$ 强相关,在门控决策中达到高 AUROC/AUPRC,实现安全、标签高效的模型替换。
原文摘要 · Abstract (English)
When a source-trained model $Q$ is replaced by a model $\tilde{Q}$ trained on shifted data, its performance on the source domain can change unpredictably. To address this, we study the two-model risk change, $ΔR := R_P(\tilde{Q}) - R_P(Q)$, under covariate shift. We introduce TRACE (Theoretical Risk Attribution under Covariate-shift Effects), a framework that decomposes $|ΔR|$ into an interpretable upper bound. This decomposition disentangles the risk change into four actionable factors: two generalization gaps, a model change penalty, and a covariate shift penalty, transforming the bound into a powerful diagnostic tool for understanding why performance has changed. To make TRACE a fully computable diagnostic, we instantiate each term. The covariate shift penalty is estimated via a model sensitivity factor (from high-quantile input gradients) and a data-shift measure; we use feature-space Optimal Transport (OT) by default and provide a robust alternative using Maximum Mean Discrepancy (MMD). The model change penalty is controlled by the average output distance between the two models on the target sample. Generalization gaps are estimated on held-out data. We validate our framework in an idealized linear regression setting, showing the TRACE bound correctly captures the scaling of the true risk difference with the magnitude of the shift. Across synthetic and vision benchmarks, TRACE diagnostics are valid and maintain a strong monotonic relationship with the true performance degradation. Crucially, we derive a deployment gate score that correlates strongly with $|ΔR|$ and achieves high AUROC/AUPRC for gating decisions, enabling safe, label-efficient model replacement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。