arXiv:2603.29725stat.MLcs.LG2026-03

解决密度比无界问题,提升分布偏移下的学习效果

Unbounded Density Ratio Estimation and Its Application to Covariate Shift Adaptation

  • 三步法估计无界密度比:相对比估计→截断→转换回标准比
  • 理论证明收敛率最优,适用于真实场景中常见的无界比值
  • 适合研究分布偏移、重要性加权的学者和工程实践者

本文聚焦于未充分研究但至关重要的无界密度比估计问题及其在协变量偏移适应中的应用。现有方法通常假设密度比有界或完全已知,这些条件在实际中常不成立,导致理论与应用脱节。本文直接处理无界密度比,将其融入重要性加权以实现有效的协变量偏移适应。提出三步估计方法:(1) 估计相对密度比;(2) 通过截断控制其无界性;(3) 将截断估计转换为标准密度比。所获密度比作为重要性权重用于协变量偏移下的回归。我们建立了非渐近的收敛性保证,证明了密度比估计器和回归函数估计器均达到最优或近似最优收敛率。研究为密度比估计与协变量偏移学习提供了新的理论洞见,将经典学习理论拓展至更贴近现实的挑战性场景。

原文摘要 · Abstract (English)

This paper focuses on the problem of unbounded density ratio estimation -- an understudied yet critical challenge in statistical learning -- and its application to covariate shift adaptation. Much of the existing literature assumes that the density ratio is either uniformly bounded or unbounded but known exactly. These conditions are often violated in practice, creating a gap between theoretical guarantees and real-world applicability. In contrast, this work directly addresses unbounded density ratios and integrates them into importance weighting for effective covariate shift adaptation. We propose a three-step estimation method that leverages unlabeled data from both the source and target distributions: (1) estimating a relative density ratio; (2) applying a truncation operation to control its unboundedness; and (3) transforming the truncated estimate back into the standard density ratio. The estimated density ratio is then employed as importance weights for regression under covariate shift. We establish rigorous, non-asymptotic convergence guarantees for both the proposed density ratio estimator and the resulting regression function estimator, demonstrating optimal or near-optimal convergence rates. Our findings offer new theoretical insights into density ratio estimation and learning under covariate shift, extending classical learning theory to more practical and challenging scenarios.

密度比分布偏移重要性加权

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。