用约束矩的方法提升神经网络去偏效果,更稳定可靠。
Automatic debiasing of neural networks via moment-constrained learning
- 通过约束预测矩来直接学习去偏所需的重构表示
- 在半合成数据上优于现有最先进方法,性能更稳健
- 适合需要高可靠性因果推断的研究者使用
经济学与生物统计学中的因果与非参数估计量常可视为未知结果回归函数上线性泛函的均值。直接学习回归函数并取样本均值会导致估计偏差,已有大量去偏研究通过额外学习目标估计量的Riesz重构表示(如目标学习、双机器学习、自动去偏等)来解决。传统方法通过推导重构表示的函数形式进行学习,但常面临极端逆概率权重或需学习条件密度等问题。近期自动去偏(AD)通过定制损失函数直接学习重构表示,但仍有局限。本文提出矩约束学习新方法,通过约束预测矩改进重构表示估计的鲁棒性,尤其对优化超参数不敏感。虽不依赖特定学习器,本文以神经网络为例,在半合成数据上评估平均处理效应与导数效应估计任务,实验显示性能优于当前最优基准。
原文摘要 · Abstract (English)
Causal and nonparametric estimands in economics and biostatistics can often be viewed as the mean of a linear functional applied to an unknown outcome regression function. Naively learning the regression function and taking a sample mean of the target functional results in biased estimators, and a rich debiasing literature has developed where one additionally learns the so-called Riesz representer (RR) of the target estimand (targeted learning, double ML, automatic debiasing etc.). Learning the RR via its derived functional form can be challenging, e.g. due to extreme inverse probability weights or the need to learn conditional density functions. Such challenges have motivated recent advances in automatic debiasing (AD), where the RR is learned directly via minimization of a bespoke loss. We propose moment-constrained learning as a new RR learning approach that addresses some shortcomings in AD, constraining the predicted moments and improving the robustness of RR estimates to optimization hyperparamters. Though our approach is not tied to a particular class of learner, we illustrate it using neural networks, and evaluate on the problems of average treatment/derivative effect estimation using semi-synthetic data. Our numerical experiments show improved performance versus state of the art benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。