解决机器学习中未观测选择偏差导致的过度自信问题
Heckman-Corrected Epistemic Uncertainty: Selection on Unobservables Defeats Importance Weighting
- 引入赫克曼模型修正未观测变量引起的样本选择偏差
- 在相关性高达0.9时，基线方法覆盖率降至64.4%，修正后达88.9%
- 适合处理真实数据中非随机缺失（MNAR）场景的不确定性建模
机器学习训练数据常通过模型无法观察到的选择过程收集：例如贷款仅在批准后可见，结果仅在测试执行后记录。标准方法（重要性加权、协变量偏移校正、MAR插补）假设选择可忽略，但1979年经济学提出的赫克曼两阶段模型通过联合拟合选择方程与结果方程，并利用逆米尔比率校正未观测变量相关的选择偏差。本文将其应用于深度认知不确定性：构建深度结果网络、线性选择头及全样本联合双正态似然，通过集成估计预测方差。在模拟生成器中，采样概率依赖于与结果噪声高度相关（rho=0.9）的未观测变量，深度集成、MC丢弃和高斯过程基线在被避开区域过度自信，名义90%置信区间覆盖率降至64.4%；即使使用已知倾向得分，重要性加权也无法修复条件偏差。赫克曼修正在具备工具变量时恢复覆盖率至88.9%，无工具变量则退化至40.3%。在真实表格数据中诱导非随机缺失（MNAR）时，该方法在选中率低区域表现最优（最低区域ECE），而其他方法仅靠过度拓宽实现表面覆盖。估算器复现经典Stata输出至七位有效数字，明确识别适用情形并开源代码。
原文摘要 · Abstract (English)
Training data for machine learning is routinely collected by a selection process the model never sees: loans are observed only when granted, outcomes only when a test was ordered. The standard fixes -- importance weighting, covariate-shift correction, MAR imputation -- assume selection is ignorable given observables. Econometrics solved the harder case in 1979: Heckman's two-equation model jointly fits a probit selection equation and an outcome equation linked through correlated errors, and the inverse-Mills-ratio term corrects for selection on unobservables, where importance weighting is structurally helpless. We instantiate this for deep epistemic uncertainty: a deep outcome network, a linear selection head, and a joint bivariate-normal likelihood over all units, ensembled for predictive variance. In a controlled generator where sampling probability depends on an unobservable correlated (rho up to 0.9) with the outcome noise, deep ensembles, MC dropout, and GP baselines are overconfident exactly where data was avoided: coverage of nominal-90% intervals falls to 64.4% at rho=0.9, and importance weighting with oracle propensities does not fix it (43.1%) -- reweighting corrects the covariate distribution, not the conditional bias E[y|x,selected] != E[y|x]. The Heckman correction restores coverage (88.9%) when the selection equation has an instrument -- a variable affecting selection but not the outcome -- and degrades measurably without one (40.3%); we chart this honesty curve rather than hide it. On real tabular data with induced MNAR selection, the corrected intervals are the best-calibrated (lowest region-ECE) non-oracle method in selected-against regions; baselines matching its raw coverage do so only by over-widening everywhere. Our estimators reproduce classic Stata output to seven digits. We state which identification regime a practitioner is in, and release the code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。