arXiv:2607.09371stat.MLcs.LG2026-07

提出非线性梯度提升的光谱去混淆方法,提升隐藏混杂下的模型稳定性。

Spectrally Deconfounded Gradient Boosting

论文配图:Spectrally Deconfounded Gradient Boosting
图 1 · 摘自论文原文
  • 用光谱损失替代平方误差,减缓混杂方向的学习速度。
  • 结合正则化与早停,实现对隐藏混杂的有效抑制。
  • 适用于一般似然与非线性混杂,比现有方法更高效可扩展。

灵活的机器学习方法易受隐藏混杂影响,可能学习到由未观测混杂因子引起的关联而非稳定信号。光谱去混淆通过收缩协变量矩阵中高方差方向(在密集混杂下携带潜在混杂信息)来缓解此问题。现有工作主要聚焦于线性模型。本文提出梯度提升的非线性光谱去混淆框架,将普通平方误差损失替换为光谱损失,改变提升动态,减缓在混杂对齐方向上的学习。我们发现,去混淆并非仅由光谱损失实现,而是光谱收缩与正则化(尤其是早停)相互作用的结果。此外,我们提供混合模型解释,将LAVA型收缩与随机效应调整关联,并推导出经验贝叶斯调参方法。还将方法扩展至一般似然和非线性混杂,使用拉普拉斯近似与核随机效应。在合成与真实数据实验中,光谱去混淆提升目标函数估计性能,且显著优于现有非线性光谱去混淆基线。

原文摘要 · Abstract (English)

Flexible machine-learning methods can be sensitive to hidden confounding: they may learn associations induced by unobserved confounders rather than stable signals. Spectral deconfounding mitigates this problem by shrinking high-variance directions of the covariate matrix that, under dense confounding, carry latent confounder information. Existing work has largely focused on linear models. We develop a nonlinear spectral deconfounding framework for gradient boosting. Our approach replaces the ordinary squared-error loss by a spectral loss, which alters the boosting dynamics by slowing down learning in confounding-aligned directions. We show that deconfounding is not achieved by the spectral loss alone, but by the interaction between spectral shrinkage and regularization, especially in terms of early stopping. Moreover, we provide a mixed-model interpretation that connects LAVA-type shrinkage to random-effects adjustment and yields an empirical-Bayes procedure for tuning the spectral loss. We also extend the method to general likelihoods and nonlinear confounding using Laplace approximations and kernel random effects. Across synthetic and real-world experiments, spectrally deconfounded boosting improves estimation of the target function under hidden confounding and is substantially more scalable than existing nonlinear spectral deconfounding baselines.

梯度提升去混淆非线性建模统计学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。