通过重要性采样修正标签分布偏移,提升部分域适应的泛化能力。
Partial Domain Adaptation via Importance Sampling-based Shift Correction
- 构建与目标域同分布的采样域,通过重要性采样挖掘潜在结构。
- 理论证明模型泛化误差可被有效控制,且采样域与源域偏移程度可解释。
- 引入基于最优传输的独立性准则,降低计算复杂度至O(n²),适合实际应用。
部分域适应(PDA)是现实机器学习中的挑战任务,旨在将有标签源域知识迁移到相关无标签目标域,其中源域标签支持集包含目标域。以往方法通过加权源域样本修正标签分布偏移,但简单重加权难以挖掘潜在结构,易导致模型在源域过拟合。本文提出一种基于重要性采样的偏移修正方法(IS²C),构建一个标签分布与目标域一致的采样域,从中生成新标注数据以刻画潜在结构并增强模型泛化能力。理论上证明了IS²C可充分控制泛化误差;通过混合分布采样,源域与采样域间的偏移程度与泛化误差直接关联,提供可解释性依据。为提升知识迁移,提出基于最优传输的条件分布对齐独立性准则,其计算复杂度可从O(n³)降至O(n²),适用于真实场景。大量实验验证了理论结果,并证明了该方法优于现有方法。
原文摘要 · Abstract (English)
Partial domain adaptation (PDA) is a challenging task in real-world machine learning scenarios. It aims to transfer knowledge from a labeled source domain to a related unlabeled target domain, where the support set of the source label distribution subsumes the target one. Previous PDA works managed to correct the label distribution shift by weighting samples in the source domain. However, the simple reweighing technique cannot explore the latent structure and sufficiently use the labeled data, and then models are prone to over-fitting on the source domain. In this work, we propose a novel importance sampling-based shift correction (IS$^2$C) method, where new labeled data are sampled from a built sampling domain, whose label distribution is supposed to be the same as the target domain, to characterize the latent structure and enhance the generalization ability of the model. We provide theoretical guarantees for IS$^2$C by proving that the generalization error can be sufficiently dominated by IS$^2$C. In particular, by implementing sampling with the mixture distribution, the extent of shift between source and sampling domains can be connected to generalization error, which provides an interpretable way to build IS$^2$C. To improve knowledge transfer, an optimal transport-based independence criterion is proposed for conditional distribution alignment, where the computation of the criterion can be adjusted to reduce the complexity from $\mathcal{O}(n^3)$ to $\mathcal{O}(n^2)$ in realistic PDA scenarios. Extensive experiments on PDA benchmarks validate the theoretical results and demonstrate the effectiveness of our IS$^2$C over existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。